I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
Cross validation and hold out sets
Validation is a popular method for estimating generalisation error in machine learning. There are many resources available that describe it (for example this FAQ), and why it is important to optimise for generalisation error and not training error. Please look these up if you are not familiar with this concept. The key idea is that we need to set some data aside that was not used for estimating the model so that we can properly measure the performance of our classifier. In particular we will use two strategies for keeping data aside:

split sample or hold out validation
cross validation

In split sample validation, a random subset of the data is held out for testing and the remaining data is used for training. To obtain multiple validation splits, the process is just repeated. For cross validation (for example 5 fold cross validation), the data is split into 5 pieces, and 4 of the pieces are used for training and 1 piece is used for validation. By considering each of the 5 pieces as a validation set in turn, we obtain 5 training/test splits that give us 5 estimates of validation error.
Conventions used in this notebook
The problem of model selection
Machine learning algorithms often have parameters that need to be tuned. In the example below, we consider the support vector machine, where we have to tune the regularisation parameter $C$ and the Gaussian kernel bandwidth $\sigma$. How do we choose the best parameters? Well, by checking which parameters performs best on a validation set! If we only have one layer of validation, then we no longer have any data left to estimate the performance of this classifier with the best parameters.
Training, validation and test sets
It quickly becomes confusing when we think about how to do model selection. When reading literature about this issue, different words are used. For convenience, we will refer to the data that is used to train a classifier as training set, the data used for model selection as validation set, and the data used to...
Can you write Python code for it?

from IPython.display import Image
Image('dummy_perf.png')


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
Interface to classifier
We have hidden away the complexity of the classifier in a separate file. In particular, there is only one common interface, which is a train_and_predict function. All this function needs to know is the dataset under consideration, the index of the training and prediction instances, the classifier to be used, and hyperparameter values (if any).
def train_and_predict(data_name, train_idx, pred_idx, method, C=None, sigma=None, verbose=False):
    """Work out which classifier to call on data_name.
    Then train the classifier, and predict on the validation set.

    return the accuracy of the classifier on the prediction set.
    """

The actual code that does the computations
We summarise the setup that we have:

manage the indices of examples to be used for training and prediction
an interface to train a classifer and predict on the prediction set

For random forest, there is no need for any hyperparameter selection, so we can directly fill in the final accuracy results.
Can you write Python code for it?

from classifier import train_and_predict

idx = pd.IndexSlice
num_ex = {'bupa': 345, 'ionosphere': 351, 'pima': 768}
for data_name in datasets:
    orig_idx = np.arange(num_ex[data_name], dtype=int)
    for split in split_idx:
        train_idx, pred_idx = random_split(orig_idx, split)
        acc = train_and_predict(data_name, train_idx, pred_idx, 'rf')
        accuracy.loc[split,idx[data_name,'rf']] = acc
print(accuracy)


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
Note that different splits could have different optimal parameters. As an important aside, it is good to reflect on what conclusions you can draw by looking at the boxplots of accuracy. There are really multiple classifiers here, and not just one single SVM. Not only are the training data different, but the parameters are different too. Now we are ready to train the final SVMs.
Can you write Python code for it?

for data_name in datasets:
    orig_idx = np.arange(num_ex[data_name], dtype=int)
    for split in split_idx:
        train_idx, pred_idx = random_split(orig_idx, split)
        C = best_param.loc[split, idx[data_name,'C']]
        sigma = best_param.loc[split, idx[data_name,'sigma']]
        acc = train_and_predict(data_name, train_idx, pred_idx, 'svm', C=C, sigma=sigma)
        accuracy.loc[split,idx[data_name,'svm']] = acc
print(accuracy)


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
Now that we have all the values for the inner loop, we can collect the best parameters for each split.
Can you write Python code for it?

def find_best_param(cv_accuracy, dataset, split):
    """Given the (inner) cross validation accuracies for all parameters,
    find the parameters that give the best average performance.
    """
    idx = pd.IndexSlice
    cv = cv_accuracy.loc[idx[split],idx[dataset]]
    avg_cv = cv.mean(axis=0)
    best_param = avg_cv.idxmax()
    return best_param

for data_name in datasets:
    for split in split_idx:
        C, sigma = find_best_param(cv_accuracy, data_name, split)
        best_param.loc[split, idx[data_name,'C']] = C
        best_param.loc[split, idx[data_name,'sigma']] = sigma
print(best_param)


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
That is all there is for the conceptual part of the double loop procedure for doing model selection with the inner loop and then using the best parameters for each split to estimate the performance on the test set. Therefore all we need to do now is implement the code to fill in the values in the dataframes appropriately.
Experimental protocol (implementation details)
Implementation of data splitting for validation
Recall that for split sample validation, a random subset of the data is held out for testing and the remaining data is used for training. To obtain multiple validation splits, the process is just repeated. For cross validation (for example 5 fold cross validation), the data is split into 5 pieces, and 4 of the pieces are used for training and 1 piece is used for validation. By considering each of the 5 pieces as a validation set in turn, we obtain 5 training/test splits that give us 5 estimates of validation error.
Can you write Python code for it?

def random_split(orig_idx, split_idx, frac_train=0.7):
    """Returns training and prediction indices which are subsets of orig_idx.
    split_idx is for reproducibility of the random permutation.

    np.random.RandomState is not portable between machines.
    """
    prng = np.random.RandomState(split_idx)
    num_data = len(orig_idx)
    num_train = int(np.round(frac_train*num_data))
    perm_idx = prng.permutation(num_data)
    train_idx = orig_idx[perm_idx[:num_train]]
    pred_idx = orig_idx[perm_idx[num_train:]]
    return train_idx, pred_idx

def cross_val(orig_idx, fold, num_cv=5):
    """Returns training and prediction indices of cross validation,
    for fold 'fold' in num_cv.
    """
    num_data = len(orig_idx)
    train = []
    pred = []
    for ix in range(num_data):
        if ix % num_cv == fold:
            pred.append(ix)
        else:
            train.append(ix)
    train_idx = orig_idx[np.array(train)]
    pred_idx = orig_idx[np.array(pred)]
    return train_idx, pred_idx

orig_idx = np.arange(20, dtype=int)
print(random_split(orig_idx, 1))
print(random_split(orig_idx, 4))
print(random_split(orig_idx, 1))
print(cross_val(orig_idx, 0))
print(cross_val(orig_idx, 3))


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
That's it!. The results of the experiment are in the above table. We can visualise this table as boxplots.
Can you write Python code for it?

fig, axes = plt.subplots(1, 3, sharey=True, figsize=(8,6))
axes[0].set_ylabel('Accuracy')
for ix,data_name in enumerate(datasets):
    ax = axes[ix]
    acc = accuracy.loc[:,data_name].values
    ax.boxplot([list(acc[:,0]), list(acc[:,1])])
    ax.set_xticklabels(methods, rotation=60)
    ax.set_title(data_name)
    ax.set_ylim(bottom=0.55, top=1.)
    ax.yaxis.grid(True)
fig.suptitle('Performance comparison of SVM and random forest')


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
The (imaginary) data used to generate this plot would be a matrix with two columns, one each for SVM and RF. Each column would contain the accuracy for each random split of the data. So for 10 splits, this means we have 10 rows. Note that this is only for one single dataset, and we would have 6 columns in total since we are considering three datasets. Pandas provides a very useful concept of DataFrames, which would be familiar to people used to R.
Can you write Python code for it?

datasets = ['bupa','ionosphere','pima']
methods = ['rf','svm']
num_splits = 10
split_idx = range(num_splits)
col_idx = pd.MultiIndex.from_product([datasets, methods], names=['dataset','method'])
# Create a dataframe with col_idx columns and split_idx rows
accuracy = pd.DataFrame(columns=col_idx, index=split_idx)
accuracy.index.name = 'split'
print(accuracy)


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
The SVM requires an inner loop for finding the best parameters
Can you write Python code for it?

for data_name in datasets:
    orig_idx = np.arange(num_ex[data_name], dtype=int)
    for split in split_idx:
        tv_idx, test_idx = random_split(orig_idx, split)
        for fold in folds:
            train_idx, pred_idx = cross_val(tv_idx, fold)
            for C in Cs:
                for sigma in sigmas:
                    acc = train_and_predict(data_name, train_idx, pred_idx, 'svm', C=C, sigma=sigma)
                    cv_accuracy.loc[idx[split,fold],idx[data_name,C,sigma]] = acc
print(cv_accuracy.head())


I am working on the file "CrossVal/expt_protocol.ipynb".
The first task is:
What needs to be done to find these best parameters? We need to fill in a table that contains the accuracy of five fold cross validation for each combination of parameters. Note that we only need to do this for SVM
Can you write Python code for it?

Cs = [0.1, 1., 10.0]
sigmas = [1e-2, 1e-1]
num_cv = 5
folds = range(5)
col_idx = pd.MultiIndex.from_product([datasets, Cs, sigmas], names=['dataset','C','sigma'])
row_idx = pd.MultiIndex.from_product([split_idx, folds], names=['split','fold'])
cv_accuracy = pd.DataFrame(columns=col_idx, index=row_idx)
print(cv_accuracy.head(13))


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
2. And we can logarithmically/geometrically re-bin a cross spectrum
In this re-binning, each bin size is 1+f times larger than the previous bin size, where f is user-specified and normally in the range 0.01-0.1. The default value is f=0.01.
Logarithmic rebinning only keeps the real part of the cross spectum.
Can you write Python code for it?

# Both of the following ways are allowed syntax:
# log_rb_cs, log_rb_freq, binning = Crossspectrum.rebin_log(avg_cs, f=0.02)
log_rb_cs = avg_cs.rebin_log(f=0.02)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
2. Energy-dependent lags
The lag vs energy spectrum can be calculated using the LagEnergySpectrum from stingray.varenergy. Refer to the Spectral Timing documentation.
Coherence
Coherence is a Fourier-frequency-dependent measure of the linear correlation between time series measured simultaneously in two energy channels.
See Vaughan and Nowak 1997, ApJ, 474, L43 and Uttley et al. 2014, A&ARev, 22, 72 section 2.1.3.
Can you write Python code for it?

long_dt = 0.03125  # seconds
long_exposure = 1600.  # seconds
long_times = np.arange(0, long_exposure, long_dt)  # seconds

long_signal_1 = 300 * np.sin(2.*np.pi*long_times/0.5) + 1000
long_signal_2 = 200 * np.sin(2.*np.pi*long_times/0.5 + np.pi/4) + 900

long_noisy_1 = np.random.poisson(long_signal_1*dt)
long_noisy_2 = np.random.poisson(long_signal_2*dt)

long_lc1 = Lightcurve(long_times, long_noisy_1)
long_lc2 = Lightcurve(long_times, long_noisy_2)

avg_cs = AveragedCrossspectrum.from_lightcurve(long_lc1, long_lc2, 8.)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
2. Pass both light curves to the AveragedCrossspectrum class with a specified segment_size.
If the exposure (length) of the light curve cannot be divided by segment_size with a remainder of zero, the last incomplete segment is thrown out, to avoid signal artefacts. Here we're using 8 second segments.
Can you write Python code for it?

avg_cs = AveragedCrossspectrum.from_lightcurve(long_lc1, long_lc2, 8.)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
2. Pass both of the light curves to the Crossspectrum class to create a Crossspectrum object.
The first Lightcurve passed is the channel of interest or interest band, and the second Lightcurve passed is the reference band.
You can also specify the optional attribute norm if you wish to normalize the real part of the cross spectrum to squared fractional rms, Leahy, or squared absolute normalization. The default normalization is 'frac'.
Can you write Python code for it?

cs = Crossspectrum.from_lightcurve(lc1, lc2)
print(cs)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
And we can plot the coherence vs the frequency.
Can you write Python code for it?

fig, ax = plt.subplots(1,1,figsize=(8,5))
# ax.hlines(0, avg_cs.freq[0], avg_cs.freq[-1], color='black', linestyle='dashed', lw=2)
ax.errorbar(avg_cs.freq, coh, yerr=err_coh, lw=2, color='blue')
ax.set_xlabel("Frequency (Hz)", fontproperties=font_prop)
ax.set_ylabel("Coherence", fontproperties=font_prop)
ax.tick_params(axis='x', labelsize=14)
ax.tick_params(axis='y', labelsize=14)
ax.tick_params(which='major', width=1.5, length=7)
ax.tick_params(which='minor', width=1.5, length=4)
for axis in ['top', 'bottom', 'left', 'right']:
    ax.spines[axis].set_linewidth(1.5)
plt.show()


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Here we plot the three normalized averaged cross spectra.
Can you write Python code for it?

fig, [ax1, ax2, ax3] = plt.subplots(3,1,figsize=(6,12))
ax1.plot(avg_cs_leahy.freq, avg_cs_leahy.power, lw=2, color='black')
ax1.set_xlabel("Frequency (Hz)", fontproperties=font_prop)
ax1.set_ylabel("Leahy cross-power", fontproperties=font_prop)
ax1.set_yscale('log')
ax1.tick_params(axis='x', labelsize=14)
ax1.tick_params(axis='y', labelsize=14)
ax1.tick_params(which='major', width=1.5, length=7)
ax1.tick_params(which='minor', width=1.5, length=4)
ax1.set_title("Leahy norm.", fontproperties=font_prop)

ax2.plot(avg_cs_frac.freq, avg_cs_frac.power, lw=2, color='black')
ax2.set_xlabel("Frequency (Hz)", fontproperties=font_prop)
ax2.set_ylabel("rms cross-power", fontproperties=font_prop)
ax2.tick_params(axis='x', labelsize=14)
ax2.tick_params(axis='y', labelsize=14)
ax2.set_yscale('log')
ax2.tick_params(which='major', width=1.5, length=7)
ax2.tick_params(which='minor', width=1.5, length=4)
ax2.set_title("Fractional rms-squared norm.", fontproperties=font_prop)

ax3.plot(avg_cs_abs.freq, avg_cs_abs.power, lw=2, color='black')
ax3.set_xlabel("Frequency (Hz)", fontproperties=font_prop)
ax3.set_ylabel("Absolute cross-power", fontproperties=font_prop)
ax3.tick_params(axis='x', labelsize=14)
ax3.tick_params(axis='y', labelsize=14)
ax3.set_yscale('log')
ax3.tick_params(which='major', width=1.5, length=7)
ax3.tick_params(which='minor', width=1.5, length=4)
ax3.set_title("Absolute rms-squared norm.", fontproperties=font_prop)

for axis in ['top', 'bottom', 'left', 'right']:
    ax1.spines[axis].set_linewidth(1.5)
    ax2.spines[axis].set_linewidth(1.5)
    ax3.spines[axis].set_linewidth(1.5)
plt.tight_layout()
plt.show()


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Here we're plotting them to see what they look like.
Can you write Python code for it?

fig, ax = plt.subplots(1,1,figsize=(10,6))
ax.plot(lc1.time, lc1.counts, lw=2, color='blue')
ax.plot(lc1.time, lc2.counts, lw=2, color='red')
ax.set_xlabel("Time (s)", fontproperties=font_prop)
ax.set_ylabel("Counts (cts)", fontproperties=font_prop)
ax.tick_params(axis='x', labelsize=16)
ax.tick_params(axis='y', labelsize=16)
ax.tick_params(which='major', width=1.5, length=7)
ax.tick_params(which='minor', width=1.5, length=4)
plt.show()


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
If m is less than 50 and you try to compute the coherence, a warning will pop up letting you know that your number of segments is significantly low, so the error on coherence might not follow the expected (Gaussian) statistical distributions.
Can you write Python code for it?

test_cs = AveragedCrossspectrum.from_lightcurve(long_lc1, long_lc2, 40.)
print(test_cs.m)
coh, err = test_cs.coherence()


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Note that also the AveragedCrossspectrum object could have been initialized using different input types:

from EventList objects as

bin_time = 0.1
   ps = AveragedCrossspectrum.from_events(
       events1, events2, dt=bin_time, segment_size=segment_size,
       norm="leahy")
   (note, again, the necessity of the bin time)

from numpy arrays of times, as
   bin_time = 0.1
   ps = AveragedCrossspectrum.from_events(
       times1, times2, dt=bin_time, segment_size=segment_size,
       gti=[[t0, t1], [t2, t3], ...], norm="leahy")
   where the light curves, uniformly binned at 0.1 s in this case, are created internally, and the good time intervals (time interval where the instrument was collecting data nominally) are passed by hand. Note that the frequencies of the cross spectrum will be expressed in inverse units as the input time arrays. If the times are expressed in seconds, frequencies will be in Hz; with times in days, frequencies will be in 1/d, and so on. We do not support units (e.g. astropy units) yet, so the user should pay attention to these details.

from iterables of light curves
   ps = AveragedCrossspectrum.from_lc_iter(
       lc_iterable1, lc_iterable2, dt=bin_time, segment_size=segment_size,
       norm="leahy")
   where lc_iterableX is any iterable of Lightcurve objects (list, tuple, generator, etc.) and dt is the sampling time of the light curves. Note that this dt is needed because the iterables might be generators, in which case the light curves are lazy-loaded after a bunch of operations using dt have been done.

Again we can print the first five Fourier frequencies and first five cross spectral values, as well as the number of segments.
Can you write Python code for it?

print(avg_cs.freq[0:5])
print(avg_cs.power[0:5])
print("\nNumber of segments: %d" % avg_cs.m)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Note that like rebin, rebin_log returns a Crossspectrum or AveragedCrossspectrum object (depending on the input object):
Can you write Python code for it?


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Note that, in principle, the Crossspectrum object could have been initialized directly as
ps = Crossspectrum(lc1, lc2, norm="leahy")
However, we recommend using the specific method for input light curve objects used above, for clarity. Equivalently, one can initialize a Crossspectrum object:

from EventList objects as

bin_time = 0.1
   ps = Crossspectrum.from_events(events1, events2, dt=bin_time, norm="leahy")
   where the light curves, uniformly binned at 0.1 s, are created internally.

from numpy arrays of times, as
   bin_time = 0.1
   ps = Crossspectrum.from_events(times1, times2, dt=bin_time, gti=[[t0, t1], [t2, t3], ...], norm="leahy")
   where the light curves, uniformly binned at 0.1 s in this case, are created internally, and the good time intervals (time interval where the instrument was collecting data nominally) are passed by hand. Note that the frequencies of the cross spectrum will be expressed in inverse units as the input time arrays. If the times are expressed in seconds, frequencies will be in Hz; with times in days, frequencies will be in 1/d, and so on. We do not support units (e.g. astropy units) yet, so the user should pay attention to these details.

from an iterable of light curves
   ps = Crossspectrum.from_lc_iter(lc_iterable1, lc_iterable2, dt=bin_time, norm="leahy")
   where lc_iterableX is any iterable of Lightcurve objects (list, tuple, generator, etc.) and dt is the sampling time of the light curves. Note that this dt is needed because the iterables might be generators, in which case the light curves are lazy-loaded after a bunch of operations using dt have been done.

We can print the first five values in the arrays of the positive Fourier frequencies and the cross power. The cross power has a real and an imaginary component.
Can you write Python code for it?


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Now let's turn noisy_1 and noisy_2 into Lightcurve objects.
Can you write Python code for it?

lc1 = Lightcurve(times, noisy_1)
lc2 = Lightcurve(times, noisy_2)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Properties
An AveragedCrossspectrum object has the following properties, same as Crossspectrum :

freq : Numpy array of mid-bin frequencies that the Fourier transform samples.
power : Numpy array of the averaged cross spectrum (complex numbers).
df : The frequency resolution (in Hz).
m : The number of cross spectra averaged together, equal to the number of whole segments in a light curve.
n : The number of data points (time bins) in one segment of the light curves.
nphots1 : The total number of photons in the first (interest) light curve.
nphots2 : The total number of photons in the second (reference) light curve.

Let's plot the amplitude of the averaged cross spectrum!
Can you write Python code for it?

avg_cs_amplitude = np.abs(avg_cs.power)

fig, ax1 = plt.subplots(1,1,figsize=(9,6))
ax1.plot(avg_cs.freq, avg_cs_amplitude, lw=2, color='blue')
ax1.set_xlabel("Frequency (Hz)", fontproperties=font_prop)
ax1.set_ylabel("Cross spectral amplitude", fontproperties=font_prop)
ax1.set_yscale('log')
ax1.tick_params(axis='x', labelsize=16)
ax1.tick_params(axis='y', labelsize=16)
ax1.tick_params(which='major', width=1.5, length=7)
ax1.tick_params(which='minor', width=1.5, length=4)
for axis in ['top', 'bottom', 'left', 'right']:
    ax1.spines[axis].set_linewidth(1.5)
plt.show()


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Re-binning a cross spectrum in frequency
Typically, rebinning is done on an averaged, normalized cross spectrum.
1. We can linearly re-bin a cross spectrum
(although this is not done much in practice)
Can you write Python code for it?

print("DF before:", avg_cs.df)
# Both of the following ways are allowed syntax:
# lin_rb_cs = Crossspectrum.rebin(avg_cs, 0.25, method='mean')
lin_rb_cs = avg_cs.rebin(0.25, method='mean')
print("DF after:", lin_rb_cs.df)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Since the negative Fourier frequencies (and their associated cross powers) are discarded, the number of time bins per segment n is twice the length of freq and power.
Can you write Python code for it?

print("Size of positive Fourier frequencies: %d" % len(cs.freq))
print("Number of data points per segment: %d" % cs.n)


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
The coherence method returns two np.ndarrays, of the coherence and uncertainty.
Can you write Python code for it?


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
Time lags / phase lags
1. Frequency-dependent lags
The lag-frequency spectrum shows the time lag between two light curves (usually non-overlapping broad energy bands) as a function of Fourier frequency.
See  Uttley et al. 2014, A&ARev, 22, 72 section 2.2.1.
Can you write Python code for it?

long_dt = 0.03125  # seconds
long_exposure = 1600.  # seconds
long_times = np.arange(0, long_exposure, long_dt)  # seconds

# long_signal_1 = 300 * np.sin(2.*np.pi*long_times/0.5) + 100 * np.sin(2.*np.pi*long_times*5 + np.pi/6) + 1000
# long_signal_2 = 200 * np.sin(2.*np.pi*long_times/0.5 + np.pi/4) + 80 * np.sin(2.*np.pi*long_times*5) + 900

long_signal_1 = (300 * np.sin(2.*np.pi*long_times*3) + 1000) * dt
long_signal_2 = (200 * np.sin(2.*np.pi*long_times*3 + np.pi/3) + 900) * dt

long_lc1 = Lightcurve(long_times, long_signal_1)
long_lc2 = Lightcurve(long_times, long_signal_2)

avg_cs = AveragedCrossspectrum.from_lightcurve(long_lc1, long_lc2, 8.)

fig, ax = plt.subplots(1,1,figsize=(10,6))
ax.plot(long_lc1.time, long_lc1.counts, lw=2, color='blue')
ax.plot(long_lc1.time, long_lc2.counts, lw=2, color='red')
ax.set_xlim(0,4)
ax.set_xlabel("Time (s)", fontproperties=font_prop)
ax.set_ylabel("Counts (cts)", fontproperties=font_prop)
ax.tick_params(axis='x', labelsize=16)
ax.tick_params(axis='y', labelsize=16)
plt.show()

fig, ax = plt.subplots(1,1,figsize=(10,6))
ax.plot(avg_cs.freq, avg_cs.power, lw=2, color='blue')
plt.show()


I am working on the file "Crossspectrum/Crossspectrum_tutorial.ipynb".
The first task is:
You'll notice that the cross spectrum is a bit noisy. This is because we're only using one segment of data. Let's try averaging together multiple segments of data.
Averaged cross spectrum example
You could use two long Lightcurves and have AveragedCrossspectrum chop them into specified segments, or give two lists of Lightcurves where each segment of Lightcurve is the same length. We'll show the first way here. Remember to check the Lightcurve tutorial notebook for fancier ways of making light curves.
1. Create two long light curves.
Generate an array of relative timestamps that's 1600 seconds long, and two signals in count rate units, with the same properties as the previous example. We then add Poisson noise and turn them into Lightcurve objects.
Can you write Python code for it?

long_dt = 0.03125  # seconds
long_exposure = 1600.  # seconds
long_times = np.arange(0, long_exposure, long_dt)  # seconds

# In count rate units here
long_signal_1 = 300 * np.sin(2.*np.pi*long_times/0.5) + 1000  # counts/s
long_signal_2 = 200 * np.sin(2.*np.pi*long_times/0.5 + np.pi/4) + 900  # counts/s

# Multiply by dt to get count units, then add Poisson noise
long_noisy_1 = np.random.poisson(long_signal_1*dt)  # counts
long_noisy_2 = np.random.poisson(long_signal_2*dt)  # counts

long_lc1 = Lightcurve(long_times, long_noisy_1)
long_lc2 = Lightcurve(long_times, long_noisy_2)

fig, ax = plt.subplots(1,1,figsize=(10,6))
ax.plot(long_lc1.time, long_lc1.counts, lw=2, color='blue')
ax.plot(long_lc1.time, long_lc2.counts, lw=2, color='red')
ax.set_xlim(0,20)
ax.set_xlabel("Time (s)", fontproperties=font_prop)
ax.set_ylabel("Counts (cts)", fontproperties=font_prop)
ax.tick_params(axis='x', labelsize=16)
ax.tick_params(axis='y', labelsize=16)
ax.tick_params(which='major', width=1.5, length=7)
ax.tick_params(which='minor', width=1.5, length=4)
plt.show()


I am working on the file "Crowd machines demo.ipynb".
The first task is:
Class frequencies
Can you write Python code for it?

from collections import Counter

Counter(spambase_data['class'].values)


I am working on the file "Crowd machines demo.ipynb".
The first task is:
Crowd machine

Run random forest with 15 or 20 different terminal node sizes, on the same training data, in each case getting the probability for each subject or instance;
Use the output from each as a new synthetic feature, which is then input to another (single) random random forest, also run in regression mode; In this case the probability estimates from each synthetic feature will be sort of continuous as they are probability estimates and not just zero or one things;
Generate some simple plots for the crowd;
Compare the crowd results to some individual random forest runs, using some two or three terminal node settings.
Can you write Python code for it?

import pandas as pd
import numpy as np
from sklearn.pipeline import make_pipeline, make_union
from sklearn.ensemble import RandomForestRegressor, RandomForestClassifier, VotingClassifier
from sklearn.feature_selection import SelectKBest
from sklearn.model_selection import cross_val_score, StratifiedKFold

spambase_data = pd.read_csv('data/spambase.tsv.gz',
                            sep='\t',
                            compression='gzip')

all_features = spambase_data.drop('class', axis=1).values
all_classes = spambase_data['class'].values

union_ops = [SelectKBest(k='all')]

for i, mwfl in enumerate(np.arange(0., 0.21, 0.01)):
    union_ops.append(VotingClassifier(estimators=[('rf-mwfl={}'.format(mwfl),
                                                   RandomForestRegressor(n_estimators=100,
                                                                         n_jobs=-1,
                                                                         min_weight_fraction_leaf=mwfl))]))

    clf = RandomForestClassifier(n_estimators=100, n_jobs=-1, min_weight_fraction_leaf=mwfl)
    print('RF w/ mwfl={:0.2f} CV score: {:0.3f}'.format(
            mwfl,
            np.mean(cross_val_score(clf, all_features, all_classes, cv=StratifiedKFold(n_splits=5, shuffle=True)))))

clf = make_pipeline(make_union(*union_ops), RandomForestClassifier(n_estimators=100, n_jobs=-1))
print('Crowd machine CV score: {:0.3f}'.format(np.mean(cross_val_score(clf, all_features, all_classes,
                                                                       cv=StratifiedKFold(n_splits=5, shuffle=True)))))


I am working on the file "Crowd machines demo.ipynb".
The first task is:
Testing data
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sb
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(spambase_data.drop('class', axis=1).values,
                                                    spambase_data['class'].values,
                                                    stratify=spambase_data['class'].values,
                                                    train_size=0.75, test_size=0.25)

clf = RandomForestRegressor(n_estimators=100, n_jobs=-1)
clf.fit(X_train, y_train)

plt.figure(figsize=(12, 7))
sb.boxplot(y_test, clf.predict(X_test))
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.xlabel('Actual status', fontsize=14)
plt.ylabel('Predicted probability', fontsize=14)
plt.ylim(-0.01, 1.01)
;


I am working on the file "Crowd machines demo.ipynb".
The first task is:
Visualize the predictions vs. actual status
Each dot corresponds to one prediction.
Training data
Can you write Python code for it?

%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sb
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(breast_cancer_data.drop('class', axis=1).values,
                                                    breast_cancer_data['class'].values,
                                                    stratify=breast_cancer_data['class'].values,
                                                    train_size=0.75, test_size=0.25)

clf = RandomForestRegressor(n_estimators=100, n_jobs=-1)
clf.fit(X_train, y_train)

plt.figure(figsize=(12, 7))
sb.swarmplot(y_train, clf.predict(X_train))
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.xlabel('Actual status', fontsize=14)
plt.ylabel('Predicted probability', fontsize=14)
plt.ylim(-0.01, 1.01)
;


%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sb
import pandas as pd
from sklearn.ensemble import RandomForestRegressor
from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(spambase_data.drop('class', axis=1).values,
                                                    spambase_data['class'].values,
                                                    stratify=spambase_data['class'].values,
                                                    train_size=0.75, test_size=0.25)

clf = RandomForestRegressor(n_estimators=100, n_jobs=-1)
clf.fit(X_train, y_train)

plt.figure(figsize=(12, 7))
sb.boxplot(y_train, clf.predict(X_train))
plt.xticks(fontsize=12)
plt.yticks(fontsize=12)
plt.xlabel('Actual status', fontsize=14)
plt.ylabel('Predicted probability', fontsize=14)
plt.ylim(-0.01, 1.01)
;


I am working on the file "CrowdDataAnalysis/analysis/plots/group_analysis_plots.ipynb".
The first task is:
China - CN: CN-01
This notebook contains few data analysis through the China's CN-01 data from Cultural Crowds dataset¹
¹ FAVARETTO, R.; DIHL, L. ; BARRETO, R. ; MUSSE, S. R. Using Group Behaviors To Detect Hofstede Cultural Dimensions. IEEE International Conference on Image Processing (ICIP), 2016.
Can you write Python code for it?

d = None
with open('..\..\group_analysis.json') as f:
    d = json.load(f)


I am working on the file "CrowdDataAnalysis/analysis/plots/group_analysis_plots.ipynb".
The first task is:
Number of groups for frame
Collected data about the number of groups through time. The average of the number of groups was 15.7777, minimum of 14 and maximum of 19.
Can you write Python code for it?

df_num_groups = pd.DataFrame(data={'Min. Num. of Groups': d['min_num_groups'], 'Avg. Num. of Groups': d['avg_num_groups'], 'Max. Num. of Groups': d['max_num_groups']})
df_num_groups

plt.figure()
ax = df_num_groups.plot(title='Number of Groups Analysis')
plt.xlabel('Frame nº')
plt.ylabel('Number of Groups')
plt.show()


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
5 Let's look at the implicit and explicit racism scores, and their sample size distributions
Most of the refs come from 4 countries (where the 4 represented leagues are). It's a shame there is no data from Italy but never mind. It's hard to know how reliable the IAT and Exp scores are with respect to the other countries, esp if there is some variation between ref biases within a country. The IAT and Exp scores might not have as much variation as within country variation..
There doesn't seem to be much 'signal' here. Also the sample sizes, though large, are then undermined by the small sample sizes for refs in some countries (data not shown, but see above)
Can you write Python code for it?

# Linked subplots showing these distributions, or a scatter plot with variable dot sizes showing how much overlap there is.
fig, ax = plt.subplots(2,2,figsize=(12, 8))
ax[0,0].hist(dfd.meanIAT.dropna().unique())
ax[0,0].set_title("Mean IAT - all")
ax[0,1].hist(dfd['meanExp'].dropna().unique())
ax[0,1].set_title("Mean Exp - all")

ax[1,0].hist(dfd_good.meanIAT.dropna().unique())
ax[1,0].set_title("Mean IAT - culled")
ax[1,1].hist(dfd_good['meanExp'].dropna().unique())
ax[1,1].set_title("Mean Exp - culled")

sns.jointplot(dfd_good['meanIAT'].dropna().unique(),dfd_good['meanExp'].dropna().unique())


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
A referee-player dyad describes the interactions between a particular ref and one player. This means that each row in the dataset is of a unique player-ref combination, listing all of the games by a given player with a particular referee at any point in his career. Let's look at the first few rows of the dataset as an example:
Can you write Python code for it?

# Display the first 10 rows of the dataset. Only 13 columns for space reasons
df.ix[:10,:13]

# Display the other columns too
df.ix[:10,13:28]


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
By default the data is in a rather counter-intuitive format: referee-player dyads. What is a dyad..?
Can you write Python code for it?

from IPython.display import HTML
HTML('<iframe src=http://en.wikipedia.org/wiki/Dyad_(sociology) width=1000 height=350></iframe>')


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
In sum, red cards are very rare. We also showed that the distributions of player and ref occurance are highly skewed. Therefore, any analysis method applied to this population needs to be able to handle these properties of the data set.
3 Discovering a major source of data impurity
How we worked out that players' entire histories are included in the data set
How many refs are there in the dataset?
Can you write Python code for it?

allRefs = dfd.refNum.value_counts()
print "Number of refs =", len(allRefs)
print "Number of dyads in the dataset =", sum(allRefs)


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
Now, that at least makes sense. We see that many England Internationals, and two World Cup winning German Internationals (each with long careers in top-flight football) have played with the same referees many times.
By contrast, how rare are red cards?
Can you write Python code for it?

print "Total interactions =", len(dfd)
clean_interactions = dfd[(dfd['allreds'] == 0)]
print "Number of red cards in the dataset =", len(dfd) - len(clean_interactions)
print "Number of interactions without a red card =", len(clean_interactions)

print "Proportion of interactions that are 'clean' =", len(clean_interactions) / float(len(dfd))


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
Please enjoy roll-over functionality for this plot!
Most games are ref'ed by someone from one of a small number of countries, as we would expect - the four premier leagues which defined selection for our data set: england, germany, france and spain.
However, it also seems like 160 different nationalities are represented by our referees. This seemed unlikely, are there really refs from almost every country on earth in the four premier leagues in the season 2012-13?
And what does the distribution of ref occurance look like?
Can you write Python code for it?

numRefs = len(dfd.refNum.value_counts())
print "Total number of referees =", numRefs
print "Median number of dyads per referee =", np.median(dfd.refNum.value_counts())

fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(12, 4))
axes.hist(dfd.refNum.value_counts().tolist(),bins=numRefs)
axes.set_xscale('symlog') # symetric log scale
axes.set_yscale('symlog')
axes.set_title("Referee occurance, log scaled")
axes.set_xlabel('log (number of occurances)')
axes.set_ylabel('log (frequency)')


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
Still 100+ nationalities represented
4 Skintone ratings could be better
We noted that ratings of skintone could be more reliable. The ratings are fairly different at the light end of the spectrum. The two raters disagree on 28742 or 19% of the time, and looking at the histograms of their responses most of these are between the first two categories. These first two categories account for ~ 70% of both rater's classifications, so biases/inconsistencies/uncertainty in this part of the dataset could have a large effect on the rest of the analysis. There could be many reasons for this, but one obvious way of dealing with it would be to use N>2 raters.
Can you write Python code for it?

# Plot of skin tone rating distributions, showing skewed nature of the data w/ histograms
# and degree of disagreement between raters with a scatterplot
rated = ((dfd['rater1']+dfd['rater2'])/2).dropna()

fig, ax = plt.subplots(1,4,figsize=(12, 4))
c = sns.color_palette()
ax[0].hist(rated,bins = 5, range = (0,1),color = c[0])
ax[0].set_title("Mean rating")

ax[1].hist(dfd['rater1'].dropna().tolist(),bins = 5, range = (0,1), color = c[1])
ax[1].set_title('Rater 1')

ax[2].hist(dfd['rater2'].dropna().tolist(),bins = 5, range = (0,1),color = c[2])
ax[2].set_title('Rater 2')

ax[3].hist((dfd['rater1'] - dfd['rater2']).dropna(), bins = 5,range = (-0.5,0.5),color = c[3])
ax[3].set_title('Difference')

fig.tight_layout()

print 'Mean skintone across the population =', np.mean(rated)


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
We already see some strange things in the data. Some refs officiated very few games. The two raters disagree about player skintone quite often. For some players there isn't a photo, so their skin tone couldn't be rated. Most dyads don't feature cards at all. In general, it's difficult to get an intuition for what the population looks like from inspecting small samples of the data. This is particularly difficult in the dyad format, as each player or ref's cards are spread un-evenly across the dataset.
2 Disaggregating the data into single player-ref interactions
Team Sheffield felt a more natural format for the data was to disaggregate it. In other words, to unpack each dyad into singular ref-player interaction. In other words, each time a player and a referee met they would contribute one row to the dataset (and thus that interaction has a maximum of one red card).
If you're interested in how the disaggregation was done, the code is in (our project folder on the OSF](https://osf.io/akqt4/) - the file is disaggregate_v3.py
Can you write Python code for it?

# Load disaggregated data
print "loading disaggregated data"
filename=os.path.join('data','crowdstorm_disaggregated.csv')
dfd = pd.read_csv(filename)


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
We lose approx 2/3rds of refs, but keep 97.4% of dyads
Can you write Python code for it?

#Copying from
#http://stackoverflow.com/questions/12065885/how-to-filter-the-dataframe-rows-of-pandas-by-within-in
#
#This line defines a new dataframe based on our >21 games filter
dfd_good=dfd[dfd['refNum'].isin(goodRefs.index.values)]

#pandas is like being in a alien spaceship
#- you know you potentially control unimaginable power, but don't know what any of the buttongs actually do
#

import mpld3
mpld3.enable_notebook()
# Histogram of country frequency.
fig, ax = plt.subplots(1,1,figsize=(12, 4))
x = dfd_good.Alpha_3.value_counts()
lines = ax.plot(x,marker='.',ms=20)

y = x.index.tolist()

tooltips = mpld3.plugins.PointLabelTooltip(lines[0], labels=y)
mpld3.plugins.connect(plt.gcf(), tooltips)

ax.set_title('Referee nationality by number of games')
ax.set_xlabel('Country number (ordered by frequency)')
ax.set_ylabel('Frequency of games')
ax.set_xlim([-3,160]) # a hack so we can see the first point most clearly

mpld3.disable_notebook()


I am working on the file "Crowdstorming_visualisation.ipynb".
The first task is:
We see that though most refs are only involved in a small number of dyads, many officiated over thousands. A median of 11 indicates that more than half of the refs officiated less than one game!
Something funny is going on - If a ref officiated a full game in one of our selected premier leagues they would be in at least 22 dyads (2 teams of 11 players each, more if substitutions occur).
Futher analysis (not shown), including a bit of eyeballing, revealled that players' entire career histories are included in the dataset. This means that if a player gets sent off in a Uruguayan league game in 2002, but then transfers to the English premier league by 2012 then this booking ends up in our dataset - as does the referee. This explains the high number of ref nationalities present in the data set
We clean the data by excluding interactions by refs who feature in fewer than 22 dyads
If you aren't in at least 22 dyads you didn't ref a game in one of our four defining leagues
Can you write Python code for it?

goodRefs = allRefs[allRefs>21]

fig, axes = plt.subplots(nrows=1, ncols=1, figsize=(12, 4))
axes.hist(goodRefs.tolist(),numRefs-11)
axes.set_xscale('symlog') # symetric log scale
plt.xlim([1,10000])
axes.set_yscale('symlog')
plt.ylim([0,1000])
axes.set_title("Referee occurance following our cull, log scaled")
axes.set_xlabel('log (number of occurances)')
axes.set_ylabel('log (frequency)')

print "Number of refs featuring in at least 22 dyads =",len(goodRefs)
print "Number of dyads, excluding refs who officiate fewer than 22 games =", sum(goodRefs)


I am working on the file "Crypto/btc/PrepareData - Technical Indicators.ipynb".
The first task is:
Plot
Can you write Python code for it?


I am working on the file "Crypto/btc/PrepareData - Technical Indicators.ipynb".
The first task is:
Price
Can you write Python code for it?

prices = dfPrices.iloc[len(dfPrices.index)-60:len(dfPrices.index)].as_matrix().ravel()
plt.figure(figsize=(25,7))
plt.plot(prices,label='Test',color='black')
plt.title('Price')
plt.legend(loc='upper left')
plt.show()


I am working on the file "Crypto/btc/PrepareData - Technical Indicators.ipynb".
The first task is:
The model in theory
We are going to use 4 features: The price itself and three extra technical indicators.
- MACD (Trend)
- Stochastics (Momentum)
- Average True Range (Volume)
Functions
Exponential Moving Average: Is a type of infinite impulse response filter that applies weighting factors which decrease exponentially. The weighting for each older datum decreases exponentially, never reaching zero.

MACD:  The Moving Average Convergence/Divergence oscillator (MACD) is one of the simplest and most effective momentum indicators available. The MACD turns two trend-following indicators, moving averages, into a momentum oscillator by subtracting the longer moving average from the shorter moving average.

Stochastics oscillator: The Stochastic Oscillator is a momentum indicator that shows the location of the close relative to the high-low range over a set number of periods.

Average True Range: Is an indicator to measure the volalitility (NOT price direction). The largest of:
- Method A: Current High less the current Low
- Method B: Current High less the previous Close (absolute value)
- Method C: Current Low less the previous Close (absolute value)

Calculation:

Can you write Python code for it?

def MACD(df,period1,period2,periodSignal):
    EMA1 = pd.DataFrame.ewm(df,span=period1).mean()
    EMA2 = pd.DataFrame.ewm(df,span=period2).mean()
    MACD = EMA1-EMA2

    Signal = pd.DataFrame.ewm(MACD,periodSignal).mean()

    Histogram = MACD-Signal

    return Histogram

def stochastics_oscillator(df,period):
    l, h = pd.DataFrame.rolling(df, period).min(), pd.DataFrame.rolling(df, period).max()
    k = 100 * (df - l) / (h - l)
    return k

def ATR(df,period):
    '''
    Method A: Current High less the current Low
    '''
    df['H-L'] = abs(df['High']-df['Low'])
    df['H-PC'] = abs(df['High']-df['Price'].shift(1))
    df['L-PC'] = abs(df['Low']-df['Price'].shift(1))
    TR = df[['H-L','H-PC','L-PC']].max(axis=1)
    return TR.to_frame()


I am working on the file "Crypto/eth/PrepareData - Technical Indicators.ipynb".
The first task is:
Average True Range
Can you write Python code for it?

atr = ATR(df.iloc[len(df.index)-60:len(df.index)],14)

plt.figure(figsize=(21,7))
#First 100 points because it's too dense
plt.plot(atr[0:100],label='ATR',color='green')
plt.title('Average True Range')
plt.legend(loc='upper left')
plt.show()


I am working on the file "Crypto/eth/PrepareData - Technical Indicators.ipynb".
The first task is:
Create complete DataFrame & Save Data
Can you write Python code for it?

dfPriceShift = dfPrices.shift(-1)
dfPriceShift.rename(columns={'Price':'PriceTarget'}, inplace=True)

dfPriceShift.head(2)

macd = MACD(dfPrices,12,26,9)
macd.rename(columns={'Price':'MACD'}, inplace=True)

stochastics = stochastics_oscillator(dfPrices,14)
stochastics.rename(columns={'Price':'Stochastics'}, inplace=True)

atr = ATR(df,14)
atr.rename(columns={0:'ATR'}, inplace=True)

final_data = pd.concat([dfPrices,dfPriceShift,macd,stochastics,atr], axis=1)
# Delete the entries with missing values (where the stochastics couldn't be computed yet) because have a lot of datapoints ;)
final_data = final_data.dropna()

final_data.info()

final_data

final_data.to_csv('BTCUSD_TechnicalIndicators.csv',index=False)


I am working on the file "Crypto/eth/PrepareData - Technical Indicators.ipynb".
The first task is:
MACD
Can you write Python code for it?

macd = MACD(dfPrices.iloc[len(dfPrices.index)-60:len(dfPrices.index)],12,26,9)

plt.figure(figsize=(25,7))
plt.plot(macd,label='macd',color='red')
plt.title('MACD')
plt.legend(loc='upper left')
plt.show()


I am working on the file "Crypto/eth/PrepareData - Technical Indicators.ipynb".
The first task is:
The model in theory
We are going to use 4 features: The price itself and three extra technical indicators.
- MACD (Trend)
- Stochastics (Momentum)
- Average True Range (Volume)
Functions
Exponential Moving Average: Is a type of infinite impulse response filter that applies weighting factors which decrease exponentially. The weighting for each older datum decreases exponentially, never reaching zero.

MACD:  The Moving Average Convergence/Divergence oscillator (MACD) is one of the simplest and most effective momentum indicators available. The MACD turns two trend-following indicators, moving averages, into a momentum oscillator by subtracting the longer moving average from the shorter moving average.

Stochastics oscillator: The Stochastic Oscillator is a momentum indicator that shows the location of the close relative to the high-low range over a set number of periods.

Average True Range: Is an indicator to measure the volalitility (NOT price direction). The largest of:
- Method A: Current High less the current Low
- Method B: Current High less the previous Close (absolute value)
- Method C: Current Low less the previous Close (absolute value)

Calculation:

Can you write Python code for it?


I am working on the file "CryptographicHashFunctions.ipynb".
The first task is:
Exemplu folosind SHA2:
Can you write Python code for it?

mesaj=hashlib.sha256('Contul meu secret dintr-un paradis financiar este 6421 1345 2974 6852')
valh=mesaj.hexdigest()
print 'Valoarea hash prin SHA2 pe 256 biti este:', valh
print 'lungimea in biti a valorii hash:', len(valh)*4

from IPython.core.display import HTML
def  css_styling():
    styles = open("./styles/custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "CryptographicHashFunctions.ipynb".
The first task is:
Sa experimentam acum in acelasi mod functia hash SHA1:
Can you write Python code for it?

m1=hashlib.sha1('anul1CTI@yahoogroups.com')
s1=m1.hexdigest()

print 'valoarea hash in hexa prin SHA1 a adresei email este', s1
print 'lungimea in biti a valorii hash este:', len(s1)*4

m1.update(' grupul anului 1')
s1n=m1.hexdigest()
print 'valoarea hash in hexa prin SHA1 a adresei email la care s-a concatenat un string este', s1n
print 'lungimea in biti a noii  valorii hash este:', len(s1n)*4


I am working on the file "CrystalCave.ipynb".
The first task is:
Calculate all possible spectras from Bchron results
Can you write Python code for it?

for it in range(chronCE.shape[1]):

    tu, Xu = coarse_grain(chronCE[:,it],Xr)
    ynew_rm = spline_interp(tu,Xu,1000,2008)
    f, psd_mt, nu = alg.multi_taper_psd(ynew_rm,Fs=1.0,adaptive=False,jackknife=False)
    psd_multi[:,it]=psd_mt

    coef,sigma=alg.AR_est_YW(ynew_rm,1)
    X_ar, noise, aph = utils.ar_generator(len(ynew_rm), sigma, coef)
    f_ar, psd_mt_ar, nu_m = alg.multi_taper_psd(X_ar ,Fs=1.0,adaptive=False,jackknife=False)
    pmtm_ar[:,it]=psd_mt_ar


I am working on the file "CrystalCave.ipynb".
The first task is:
FDR procedure
Can you write Python code for it?

#start FDR procedure
pvalr_975 = FloatVector(pval_975)
pvalr_025 = FloatVector(pval_025)
pvalr_med = FloatVector(pval_med)

r.source("fdr.R")
#sig_975 = r.fdr(pvalr_975,method="original",adjustment_method="mean")
#sig_025 = r.fdr(pvalr_025,method="original",adjustment_method="mean")
#sig_med = r.fdr(pvalr_med,method="original",adjustment_method="mean")

sig_975 = r.fdr(pvalr_975,method="original")
sig_025 = r.fdr(pvalr_025,method="original")
sig_med = r.fdr(pvalr_med,method="original")

#print(sig_975)
#prepare all global grids
latmedian=latt[:]
lonmedian=lont[:]
lat975=latt[:]
lon975=lont[:]
lat025=latt[:]
lon025=lont[:]

#delete grids which are significant
if sig_975:
    for isig in sorted(sig_975,reverse=True):
         del lat975[isig-1]
         del lon975[isig-1]
#        lat975.append(latt[isig-1])
#        lon975.append(lont[isig-1])

if sig_025:
    for isig in sorted(sig_025,reverse=True):
        del lat025[isig-1]
        del lon025[isig-1]

if sig_med:
    for isig in sorted(sig_med,reverse=True):
        del latmedian[isig-1]
        del lonmedian[isig-1]


I am working on the file "CrystalCave.ipynb".
The first task is:
Generate different age-depth realizations by using Bchron age model
Can you write Python code for it?

Bchron=importr('Bchron')
haha = Bchron.Bchronology(ages=ages, ageSds=sd, positions=positions, iterations=10000, burn=2000, thin=8, extractDate=topDate, calCurves=calCurves, predictPositions=predictPositions)


I am working on the file "CrystalCave.ipynb".
The first task is:
Get the annual mean SST from 1857-2007
Can you write Python code for it?

#annual mean
for i in range(sst_ann.shape[0]):
    sst_ann[i,:,:]=np.mean(sst[i*12+19:(i+1)*12+19,:,:],axis=0)

#sst_ann_fil=filter(sst_ann,1.0/30,0)
sst_ann_fil=sst_ann


I am working on the file "CrystalCave.ipynb".
The first task is:
Interpolate $\delta^{18}O$ on the annual scale
Can you write Python code for it?

#interpolation
d18O_multi=np.zeros((chronCE.shape[1],sst_ann_fil.shape[0]))
sst_corr=np.zeros((chronCE.shape[1],nlat,nlon))

for it in range(chronCE.shape[1]):
    tu, Xu = coarse_grain(chronCE[:,it],Xr)
    #interpolation
    tck = interpolate.splrep(tu,Xu,s=1)
    xnew = range(1857,2008,1)
    ynew = interpolate.splev(xnew,tck)
    d18O_multi[it,:] = ynew

#interpolation of the median series
tu, Xu = coarse_grain(chronQ[:,1],Xr)
tck = interpolate.splrep(tu,Xu,s=1)
xnew = range(1857,2008,1)
ynew = interpolate.splev(xnew,tck)
d18O_median = ynew

d18O_fil=d18O_multi
d18O_median_fil=d18O_median


I am working on the file "CrystalCave.ipynb".
The first task is:
Load chronology information
Can you write Python code for it?

age_c = f1["Crystal2013.chron1measurement1.csv"]["230Th Age (yr BP) (corrected)"]
age_c = np.asarray(age_c)
age_c = np.insert(age_c,0,-57) #insert the top age

depth_age = f1["Crystal2013.chron1measurement1.csv"]["Depth (mm)"]
depth_age = np.asarray(depth_age)
depth_age = np.insert(depth_age,0,0.05)

age_sd = f1["Crystal2013.chron1measurement1.csv"]["230Th Age_uncertaity (yr BP) (corrected)"]
age_sd = np.asarray(age_sd)
age_sd = np.insert(age_sd,0,0.1)


I am working on the file "CrystalCave.ipynb".
The first task is:
Load data (in LiPD format)
Can you write Python code for it?

loadLipds()
f1 = getCsv("Crystal2013.lpd")
d18O = f1["Crystal2013.paleo1measurement1.csv"]["d18O_VPDB"]
d18O = np.asarray(d18O)
depth = f1["Crystal2013.paleo1measurement1.csv"]["Depth"]
depth = np.asarray(depth)
depth = depth /10.0 #depth in cm


I am working on the file "CrystalCave.ipynb".
The first task is:
Plot the age model
Can you write Python code for it?

# PLOT IT OUT
x = depth_horizons*10
f2 = plt.figure(figsize=(10,8))
ax = plt.axes()
ax.spines["top"].set_visible(False)
ax.spines["right"].set_visible(False)
ax.get_xaxis().tick_bottom()
ax.get_yaxis().tick_left()
plt.fill_between(x, chronQ[:,0], chronQ[:,2], facecolor='Silver',edgecolor='Silver',lw=0.0)  # see http://stackoverflow.com/questions/14143092/why-does-matplotlib-fill-between-draw-edgelines-only-on-a-pdf
CI = mpatches.Patch(color='silver') # create proxy artist for labeling
lbl = ('95\% CI','median','U/Th dates','sample paths')
dat = plt.errorbar(depth_age,year_CE,age_sd,color='Crimson',fmt='o')
med, = plt.plot(x,chronQ[:,1],color = 'black', lw=3.0)  # that comma is really important !! wouldn't work without it

# plot a few random paths
nl = 10
idx = np.random.randint(nchrons+1, size=nl)
l = plt.plot(x,chronCE[:,idx],lw=0.5,color = 'Crimson')

#col = plt.cm.YlGn(np.arange(nl))
#for i in range(nl):
#    plt.plot(x,chronCE[:,idx[i]],lw=1,color=col[i,:])
#    plt.draw()
# I do not understand why line colors do not get updated. this is most irritating

lg = plt.legend((CI,med,dat,l[1]),lbl,loc='upper right'); lg.draw_frame(False)
plt.grid(axis='y'); plt.ylim(750,2020);  plt.xlim(0,120)
plt.xlabel(r'Depth (mm)',fontsize=14); plt.ylabel(r'Year (CE)',fontsize=14);
plt.title(r'Crystal BChron sampling paths, 10 dates',fontsize=16)

#f2.savefig('Crystal_BChron.pdf', dpi=400, facecolor='w', edgecolor='w',transparent=True)

# save to file
np.savez('Crystal_Bchron.npz',chronCE=chronCE,depth_horizons=depth_horizons)


I am working on the file "CrystalCave.ipynb".
The first task is:
Retrieve the result of estimated ages
Can you write Python code for it?

# The variable 'theta' is the posterior estimated values of the ages, and is stored as
# the first matrix of the List Vector class 'ages'

theta = haha[0]
theta=np.array(theta)

thetaPredict = haha[4]
thetaPredict=np.array(thetaPredict)

# Save depth horizons
depths=np.array(predictPositions)
depth_horizons=depths[:-1]

# save all the chronologies as independent time axes and plot original data with
# varying X values.

thetaPredict.shape
chrons=thetaPredict[:,:-1]      # reshape to be same size as input data

# Call BCHRON observation model to return CHRONS (in years BP)
# NOTE" Bchron will FREAK OUT unless ages increase with depth (no reversals)
chronBP = chrons

# recast in years CE
chronCE = np.flipud(1950 - chronBP).transpose()
chronQ  = mquantiles(chronCE, prob=[0.025, 0.5, 0.975], axis=1)
nchrons = chronCE.shape[1]


I am working on the file "CrystalCave.ipynb".
The first task is:
Test significance of the calculated correlations (replace the Nsample to your raw sample size)
Can you write Python code for it?

#t-test for correlation
corrQ975= corrQ_new[:,:,2]
corrQ025= corrQ_new[:,:,0]
corrQ050= corrQ_new[:,:,1]

d18O_coef, d18O_sigma = alg.AR_est_YW(d18O_median_ano,1)
neff_array=sst_ano[:,:,0]
latt,lont=[],[]
pval_025=[]
pval_975=[]
pval_med=[]

Nsample = 151

for ilat in range(nlat):
    for ilon in range(nlon):
        if np.isnan(sst_ano[ilat,ilon,0])==False:
            coef, sigma = alg.AR_est_YW(sst_ano[ilat,ilon,:],1)
#            sst_coef[ilat,ilon] = coef
            neff=Nsample*(1-d18O_coef*coef)/(1+d18O_coef*coef)
#            neff=151
            latt.append(lat0[ilat])
            lont.append(lon0[ilon])

            tval=corrQ975[ilat,ilon]/np.sqrt(1-corrQ975[ilat,ilon]**2)*np.sqrt(neff-2)
            pval0=t.sf(abs(tval),neff-2)*2
            pval_975.append(pval0)

            tval=corrQ025[ilat,ilon]/np.sqrt(1-corrQ025[ilat,ilon]**2)*np.sqrt(neff-2)
            pval0=t.sf(abs(tval),neff-2)*2
            pval_025.append(pval0)

            tval=corrQ050[ilat,ilon]/np.sqrt(1-corrQ050[ilat,ilon]**2)*np.sqrt(neff-2)
            pval0=t.sf(abs(tval),neff-2)*2
            pval_med.append(pval0)


I am working on the file "CrystalCave.ipynb".
The first task is:
plot raw age model
Can you write Python code for it?

# plot age model
plt.rc('text', usetex=True,)
plt.rc('font', family='sans-serif')

# create figure
f, (ax1, ax2) = plt.subplots(2, 1)
f.set_figheight(9); f.set_figwidth(8)
ax1.spines["top"].set_visible(False)
ax1.spines["right"].set_visible(False)
ax1.errorbar(depth_age,age_c,age_sd)
ax1.set_xlabel(r'Depth (mm)')
ax1.set_ylabel(r'Age (years since 2009, 2$\sigma$ errors)')
ax1.set_title(r'Crystal Raw age model, 10 dates')
plt.subplots_adjust(hspace = 0.4)
ax2.spines["top"].set_visible(False)
ax2.spines["right"].set_visible(False)
ax2.errorbar(depth_age,age_c,age_sd,color='Crimson')
ax2.set_xlabel(r'Depth (mm)')
ax2.set_ylabel(r'Age (years since 2009, 2$\sigma$ errors)')
ax2.set_title(r'Age model respecting the principle of superposition (10 dates)')


I am working on the file "CrystalCave.ipynb".
The first task is:
plot timeseries quantiles
Can you write Python code for it?

f, ax = plt.subplots(1, 1)
f.set_figheight(6); f.set_figwidth(8)
chronQ  = mquantiles(chronCE, prob=[0.025, 0.5, 0.975], axis=1)
Xr = d18O[:-1]
hp,= plt.plot(chronQ[:,0], Xr, lw=0.5, color='silver',ls='dashed')
hl,= plt.plot(chronQ[:,2], Xr, lw=0.5, color='silver')
hm,= plt.plot(chronQ[:,1], Xr, lw=1.0, color='DarkBlue')
plt.xlim(750,2020); plt.ylim(-7.5,-11); plt.grid(axis='y')
lbl = ('97.5\% quantile','median','2.5\% quantile')
lg = plt.legend((hp,hm,hl),lbl,loc='upper right',fontsize=9); lg.draw_frame(False)
# label axes
plt.ylabel(r'$\delta^{18}\mathrm{O}$',fontsize=12)
plt.xlabel(r'Year (CE)',fontsize=12)
plt.title(r'Crystal $\delta^{18}\mathrm{O}$ Bchron ensemble',fontsize=14)
#f.savefig('Crystal_ts.pdf', dpi=400, facecolor='w', edgecolor='w',transparent=True)


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Atencion para los usuarios de R: referencia o asignacion?
Can you write Python code for it?

x = [1,2,3] # asignacion

y = [0, x] # referencia

y

x[0] = -1 # asignamos otra lista a x

y


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Booleanos, short circuits
Can you write Python code for it?

True and True

True and False

True or True

True or False

2 > 3 or 1 < 2

(2 > 3) and (1 < 2)


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Condicionales y Loops, For, While, If, Elif
El truco para hacer loops en python es la funcion range
Can you write Python code for it?

range(1000)

for i in range(5): # for (i in 0:(n-1))
    print(i)

for i in range(10):
    if i % 2 == 0: # operador de modulo (residuo tras division)
        print(str(i) + ' par') # str convierte numero en string
    else:
        print(str(i) + ' impar')

i = 0
while i < 10:
    print(i)
    i = i + 1


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Listas
Son elementos que pueden cambiar de valores y de tamano una vez creados, son las estructuras mas usadas. En vez de usar parentesis usan brackets.
Can you write Python code for it?

x = [1,2,3]

x[0] = 'hola'

x

x.append('nuevo valor')

x

x.insert(2, 'valor intermedio')

x.pop(0)

x


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Que es mas rapido, tuplas o listas?
Can you write Python code for it?

import timeit
timeit.timeit('x = (1,2,3,4,5,6)')

timeit.timeit('x=[1,2,3,4,5,6]')


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Sets
son conjuntos matematicos
Can you write Python code for it?

A = {1,2,3}
B = {2,3,4}

A | B # union

A & B # interseccion

A - B # diferencia de conjuntos

B - A

A ^ B # diferencia simetrica


I am working on the file "CuadernosPython/.ipynb_checkpoints/2._estructuras-checkpoint.ipynb".
The first task is:
Una vez que guardan en una tupla datos, pueden accesarlos con indices en brackets. En python los indices comienzan en cero.
Can you write Python code for it?


I am working on the file "CuadernosPython/.ipynb_checkpoints/3._clases_y_herencia-checkpoint.ipynb".
The first task is:
regresando al ejemplo
Can you write Python code for it?

class Persona:
    def __init__(self, nombre, edad, altura, peso):
        self.nombre = nombre
        self.edad = edad
        self.altura = altura
        self.peso = peso
        self.gordura = peso / altura

juan = Persona("Juan Perez", 30, 175, 80)

juan.__dict__

juan.peso = 76

juan.__dict__


I am working on the file "CuadernosPython/2._estructuras.ipynb".
The first task is:
Booleanos, short circuits
Can you write Python code for it?


I am working on the file "CuadernosPython/2._estructuras.ipynb".
The first task is:
Clases
Can you write Python code for it?

class Person:
    def __init__(self, first, last):
        self.first = first # campos
        self.last = last
    def greet(self, add_msg = ""): # mas metodos
        print('hello ' + self.first + ' ' + add_msg)

juan = Person('juan', 'dominguez')

type(juan)

juan.first

juan.last

pedro = Person('pedro', 'gonzalez')

pedro.first

pedro.last

pedro.greet('my friend')

pedro.greet()


I am working on the file "CuadernosPython/2._estructuras.ipynb".
The first task is:
Diccionarios
En una GRAN cantidad de problemas, quieren almacenar claves y asignarle a cada clave un valor. Un mejor nombre para diccionario (ejemplo jaja) es "directorio telefonico".
Can you write Python code for it?

dir_tel = {'juan': 553423523,  'pedro': 243252345, 'itam':'is fun'}

dir_tel['juan']

dir_tel['itam']

dir_tel.keys()

dir_tel.values()


I am working on the file "CuadernosPython/2._estructuras.ipynb".
The first task is:
Listas
Son elementos que pueden cambiar de valores y de tamano una vez creados, son las estructuras mas usadas. En vez de usar parentesis usan brackets.
Can you write Python code for it?


I am working on the file "CuadernosPython/2._estructuras.ipynb".
The first task is:
Una vez que guardan en una tupla datos, pueden accesarlos con indices en brackets. En python los indices comienzan en cero.
Can you write Python code for it?


I am working on the file "CuboidalE3.ipynb".
The first task is:
... and simply divide by 120.
Can you write Python code for it?

E_vol = (RT3_vol * ø_down ** 3)/120 # a little more than 1/24, volume of T module
print(E_vol)


I am working on the file "CuboidalE3.ipynb".
The first task is:
Another expression (in Python) for the E's volume is (rt2(2)/8) * (φ ** -3).
For exercise, lets computer these expressions for E's and SuperRT's volume to a hundred places of decimal precision.
Can you write Python code for it?

from decimal import Decimal, localcontext
with localcontext() as cm:
    cm.prec = 300 # 100 decimal points of precision
    Phi = (Decimal(1) + Decimal(5).sqrt())/Decimal(2)
    RT3_vol = Decimal('15') * Decimal('2').sqrt()
    check_RT = Decimal('20') * (Decimal('9')/Decimal('8')).sqrt()
    E_volume = (Decimal('2').sqrt() / Decimal('8')) * Phi ** -3  # simplified expression
    check_E = (RT3_vol * Phi ** -3) / Decimal('120')  # shrink SuperRT and explode
print("E3*120  {:80.78f}".format(RT3_vol))
print("Check:  {:80.78f}".format(check_RT))
print("E vol:   {:80.78f}".format(E_volume))
print("Check:   {:80.78f}".format(check_E))


I am working on the file "CuboidalE3.ipynb".
The first task is:
How about we redo those calcs with gmpy2, why not?
Can you write Python code for it?

import gmpy2

gmpy2.get_context().precision=300
Ø = (1 + gmpy2.root(5, 2))/2
vol_scale_factor = 1.5
rad_scale_factor = gmpy2.root(vol_scale_factor, 3)

S3 = gmpy2.root(9/8, 2)
SuperRT_vol = S3 * 20
Emod_RT_vol = SuperRT_vol * (1/Ø)**3
print("E3*120: {:80.78f}".format( SuperRT_vol))
print("E vol:   {:80.78f}".format( Emod_RT_vol/120 ))


I am working on the file "Cufflinks Tutorial - Colors.ipynb".
The first task is:
Color Generators
A color generator can be used to produce shades of colors in an iterative form. For example when plotting N timeseries so the color used are as distinctive as possible.
Can you write Python code for it?

# Create a generator using 3 defined base colors
colors=['green','orange','blue']
gen=cf.colors.colorgen(colors)
outputColors=[next(gen) for _ in range(15)]
cf.colors.color_table(outputColors)

# Create a generator with default set of colors
gen=cf.colors.colorgen()
outputColors=[next(gen) for _ in range(15)]
cf.colors.color_table(outputColors)

# We can see all available scales with
cf.get_scales()

# Other color scales can be also seen here
cf.colors.scales()

colorscale=cf.colors.get_scales('accent')
cf.colors.color_table(colorscale)


I am working on the file "Cufflinks Tutorial - Colors.ipynb".
The first task is:
Color Ranges
A range of colors can be generated using a base color and varying the saturation.
Can you write Python code for it?

# 10 different tones of pink
cf.colors.color_range('pink',10)


I am working on the file "Cufflinks Tutorial - Colors.ipynb".
The first task is:
Color Tables
This function is meant to be used in an iPython Notebook.
It generates an HTML table to display either a defined list of colors or to automatically generate a range of colors.
Can you write Python code for it?

# Displaying a table of defined colors (list)
colors=['#f08', 'rgb(240, 178, 185)', 'blue' , '#32ab60']
cf.colors.color_table(colors)

# Generating 15 shades of orange
cf.colors.color_table('orange',15)


I am working on the file "Cufflinks Tutorial - Colors.ipynb".
The first task is:
Colors can be represented as strings:
HEX "#db4052"
RGB "rgb(219, 64, 82)"
RGBA "rgba(219, 64, 82, 1.0)"
Color Conversions
Can you write Python code for it?

# The colors module includes a pre-defined set of commonly used colors
cf.colors.cnames

# HEX to RGB
cf.colors.hex_to_rgb('red')

# RGB to HEX
cf.colors.rgb_to_hex('rgb(219, 64, 82)')

# RGB or HEX to RGBA (transparency)
cf.colors.to_rgba('#3780bf',.5), cf.colors.to_rgba('rgb(219, 64, 82)',.4)

# RGBA to RGB (flatten transparency)
# By default assumes that the transparency color is *white*, however this can be also passed as a parameter.
cf.colors.rgba_to_rgb('rgba(219, 64, 82, 0.4)','white')


I am working on the file "Curso_CFD/Hoja_2.ipynb".
The first task is:
Calcula ahora la matriz traspuesta y la matriz inversa de la matriz D
Can you write Python code for it?


I am working on the file "Curso_CFD/Hoja_2.ipynb".
The first task is:
Ejercicio 2
Sigue los pasos del notebook Convección con Algebra Lineal para montar el sistema $u^{n+1} = D \cdot u^{n}$. Aplícale el algoritmo upwind ( $u_i^{n+1} = u_i^n - \mathrm{Co} (u_{i}^n-u_{i-1}^n)$ ) para obtener una solución. ¿Qué es lo que pasa? ¿Es la misma solución que la que obteníamos a través de bucles?
Can you write Python code for it?


I am working on the file "Curso_CFD/Hoja_2.ipynb".
The first task is:
Hasta ahora, hemos trabajado con vectores, aplicándolos a la resolución de la ecuación de convección en 1-D. Vamos a ver ahora cómo resolver la ecuación de difusión y cómo aplicar matrices y operaciones de álgebra lineal para resolver un esquema implicito.
Ejercicio 1
Escribe una matriz diagonal D que tenga elementos igual a 2 por encima y por debajo de la diagonal principal
Can you write Python code for it?


I am working on the file "Curso_CFD/Hoja_2.ipynb".
The first task is:
Pista: las condiciones periódicas implican que $u_{n_x} = u_0$
Ejercicio 3
Por lo que parece, no supone un gran cambio utilizar matrices en lugar de bucles, y la complejidad aumenta. ¿Por qué utilizarlas entonces? Bien, porque hay ciertos esquemas que sólo pueden resolverse de esta manera: los métodos implicitos.
En un método implícito utilizamos los datos correspondientes al paso temporal siguiente. Por tanto, hay que resolver un sistema del tipo $u^{n+1} \cdot D' = u^n$, donde la matriz $D'$ es una matriz diferente a D y sólo se conserva la notación por comodidad.
Repite la solución del ejercicio anterior, pero utilizando ahora un método implicito. ¿Qué cambios observas?
Can you write Python code for it?


I am working on the file "CurvatureMatrix.ipynb".
The first task is:
Plotting
Can you write Python code for it?

fig = plt.figure()

ax1 = fig.add_subplot(141)
sf.nice_plot(means[0,:,:]/1000.,xmin,xmax,xint,
             center[0],center[1],stations_local,color='inferno_r',cmin=0,cmax=3,levels_t=(0.05,0.1,0.5,1,5))
plt.title('X RMSE')
ax2 = fig.add_subplot(142)
sf.nice_plot(means[1,:,:]/1000.,xmin,xmax,xint,
             center[0],center[1],stations_local,color='inferno_r',cmin=0,cmax=3,levels_t=(0.05,0.1,0.5,1,5))
plt.title('Y RMSE')
ax3 = fig.add_subplot(143)
sf.nice_plot(means[2,:,:]/1000.,xmin,xmax,xint,
             center[0],center[1],stations_local,color='inferno_r',cmin=0,cmax=3,levels_t=(0.05,0.1,0.5,1,5))
plt.title('Z RMSE')
ax4 = fig.add_subplot(144)
sf.nice_plot(means[3,:,:]/1000.,xmin,xmax,xint,
             center[0],center[1],stations_local,color='inferno_r',cmin=0,cmax=3,levels_t=(0.05,0.1,0.5,1,5))
plt.title('CT RMSE')

plt.show()

sf.nice_plot(means[2,:,:]/1000.,xmin,xmax,xint,
             center[0],center[1],stations_local,color='inferno_r',cmin=0,cmax=3,levels_t=(0.05,0.1,0.5,1,5))


I am working on the file "CurvatureMatrix.ipynb".
The first task is:
Set source power and run the curvature matrix math
Can you write Python code for it?

# power = 0.84 # 98% FDE or OK/COLMA 95% in Watts
# power = 1.57 # NALMA 95%
# power = 1.36  # 95% FDE
# power = 2.67  # 90% FDE
# power = 4.34 # 85% FDE
# power = 6.79 # 80% FDE
power = 9.91  # 75% FDE
# power = 14.00 # 70% FDE
# power = 10000  # High powered source for full station contribution in domain

means = np.empty((np.shape(points)[0],4))
for i in range(np.shape(points)[0]):
    means[i] = sf.curvature_matrix(points[i],stations_local,ordered_threshs,c0,power=power,timing_error=dt_rms,min_stations=6)

means = means.T.reshape((4,np.shape(initial_points)[1],np.shape(initial_points)[2]))
means  = np.ma.masked_where(np.isnan(means) , means)


I am working on the file "CurvatureMatrix.ipynb".
The first task is:
Set up the grid of source points
Can you write Python code for it?

xmin, xmax, xint = -200001, 199999, 5000 # Extent of grid (km)
alts = np.array([7000])

initial_points = np.array(np.meshgrid(np.arange(xmin,xmax+xint,xint),
                                      np.arange(xmin,xmax+xint,xint), alts))

points = initial_points.reshape(3,int(np.size(initial_points)/3)).T


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
Clearly the tail is a bit off, the next iteration is to average the tail end and use that as the y shift instead of using just the last point.
Can you write Python code for it?

yshift = np.average(ydata[-20:])
yshift

ylog = np.log(ydata[:25] - yshift)
fitlog = np.polyfit(x[:25], ylog[:25], 1,full=True)
flog = np.poly1d(fitlog[0])
plt.plot(x[:25],ylog)
plt.plot(x[:25],flog(x[:25]))

ylogfit = np.exp(flog(x))
plt.plot(x,ylogfit+yshift)
plt.plot(x,ydata)


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
Create a pure (i.e. exact) set of data with some parameters, and then simulate some data of the same system (by adding random noise).
Can you write Python code for it?

y = func(x, 2.5, 0.6, 0.5)
ydata = y + 0.2 * np.random.normal(size=len(x))


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
Looks pretty good as far as fits go. Let's check out the error:
Can you write Python code for it?

plt.plot(x,((yfit-y)/y)*100)
plt.title("Fit error %")


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
Next we define x as a linear space with 100 points that range from 0 to 10.
Can you write Python code for it?


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
Now to finally back out the exponential from the linear fit:
Can you write Python code for it?

ylogfit = np.exp(flog(x))
plt.plot(x,ylogfit+ydata[-1])
plt.plot(x,ydata)


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
The data is pretty clearly linear, but we can fit a line to determine the slope. A 1st order polynomial is a line, so we use polyfit:
Can you write Python code for it?

# execute the fit on the data; a 1-dim fit (line)
fit = np.polyfit(x, y, 1,full=True)


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
The fit is stored in a variable called fit which has several elements. We can print them out with nice labels using the following cell:
Can you write Python code for it?

print("coeffients:", fit[0])
print("residuals:", fit[1])
print("rank:", fit[2])
print("singular_values:", fit[3])
print("rcond:", fit[4])


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
The main thing we want is the list of coefficients. These are the values in the polynomial that was a best fit. We can create a function (called f) that is the best fit polynomial. Then it is easy to plot both together and see that the fit is reasonable.
Can you write Python code for it?

f = np.poly1d(fit[0])  # create a function using the fit parameters

plt.plot(x,y)
plt.plot(x,f(x))


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
Then define a function that we expect models our system. In this case, exponential decay with an offset.
Can you write Python code for it?

def func(x, a, b, c):
    return a * np.exp(-b * x) + c


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
To further illustrate the variation in this fit, repeat all the cells (to get new random noise in the data) and you'll see the fit changes. Sometimes, the error is as large as 10%. Compare this to a linear fit of log data and I bet you see much less variation in the fit!
Modeling by rescaling data
The "fit a line to anything" approach

"With a small enough data set, you can always fit it to a line"
Can you write Python code for it?

ylog = np.log(ydata[:25] - ydata[-1])
plt.plot(x[:25],ylog,".")

fitlog = np.polyfit(x[:25], ylog[:25], 1,full=True)

fitlog

ylog.shape

flog = np.poly1d(fitlog[0])
plt.plot(x[:25],ylog)
plt.plot(x[:25],flog(x[:25]))


I am working on the file "Curve Fitting/Curve fitting.ipynb".
The first task is:
y is mock data that we create by linear function with a slope of 1.45. We also add a small amount of random data to simulate noise as if this were a measured quantity.
Can you write Python code for it?

y = 1.45 * x + 1.3*np.random.random(len(x))

plt.plot(x,y,".")


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
$h = 0.008 m $
Can you write Python code for it?

h = .008 # data.csv, in meters (m) (EY : ???)

ydata = np.array(df.lift[df.h == h])
xdata = np.array(df.v[df.h == h])
print (len(ydata)); print(len(xdata))

# Calculate and show fit parameters.
popt, pcov = curve_fit(log_func_fit, xdata[3:], ydata[3:] , maxfev=100000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(3,150,200)
curvey=log_func_fit(curvex, *popt  )
plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')

plt.xlabel('$v$')
plt.ylabel('lift')
plt.title('$h=0.008$ m')

plt.show()


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
$h=0.002 \, m$
Can you write Python code for it?

df.columns

h = .002 # data.csv, in meters (m) (EY : ???)

ydata = np.array( df.ix[df.h==h]["Force2.Force_y"] )
xdata = np.array(df.v[df.h == h])
print (len(ydata)); print(len(xdata))

# Calculate and show fit parameters.
popt, pcov = curve_fit( exp_fit, xdata[3:], ydata[3:] , p0=[100,100,1,1],maxfev=1000000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(3,150,200)
curvey= exp_fit(curvex, *popt  )
plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')

plt.xlabel('$v$')
plt.ylabel('Force2.Force_y')
plt.title('$h=0.002$ m')

plt.show()


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
$h=0.032 m$
Can you write Python code for it?

h = .032 # data.csv, in meters (m) (EY : ???)

ydata = np.array(df.drag[df.h == h])
xdata = np.array(df.v[df.h == h])
print (len(ydata)); print(len(xdata))

# Calculate and show fit parameters.
popt, pcov = curve_fit(gamma_func_fit, xdata[2:], ydata[2:] , maxfev=100000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(-5,150,200)
curvey=gamma_func_fit(curvex, *popt  )
plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')

plt.xlabel('xdata')
plt.ylabel('ydata')

plt.show()


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
Fit with decaying exponentials
Can you write Python code for it?

xdata = np.array( DF1_A57["Time [ms]"] )
ydata = np.array( DF1_A57["Force1.Force_x [newton]"] )

# Calculate and show fit parameters.
popt, pcov = curve_fit( exp_fit, xdata[0:], ydata[0:] , p0=[100,100,1,-25.],maxfev=1000000)
#popt, pcov = curve_fit( exp_fit, xdata[2:], ydata[2:] , maxfev=100000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(3,510,1000)
curvey= exp_fit(curvex, *popt  )
plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')

plt.xlabel('$t [ms]$')
plt.ylabel('Force1.Force_x [newton]')
plt.title('$DF1$ ')

plt.show()

xdata = np.array( DF1_A57["Time [ms]"] )
ydata = np.array( DF1_A57["Force1.Force_y [newton]"] )

# Calculate and show fit parameters.
popt, pcov = curve_fit( exp_fit, xdata[0:], ydata[0:] , p0=[100,100,1,-25.],maxfev=1000000)
#popt, pcov = curve_fit( exp_fit, xdata[2:], ydata[2:] , maxfev=100000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(3,510,1000)
curvey= exp_fit(curvex, *popt  )
plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')

plt.xlabel('$t [ms]$')
plt.ylabel('Force1.Force_y [newton]')
plt.title('$DF1$ ')

plt.show()

xdata = np.array( DF1_A57["Time [ms]"] )
ydata = np.array( DF1_A57["Force1.Force_z [newton]"] )

# Calculate and show fit parameters.
popt, pcov = curve_fit( exp_fit, xdata[0:], ydata[0:] , p0=[1000,100,1,-25.],maxfev=1000000)
#popt, pcov = curve_fit( exp_fit, xdata[2:], ydata[2:] , maxfev=100000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(3,510,1000)
curvey= exp_fit(curvex, *popt  )
plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')

plt.xlabel('$t [ms]$')
plt.ylabel('Force1.Force_z [newton]')
plt.title('$DF1$ ')

plt.show()

xdata = np.array( XY1_A57["Time [ms]"] )
ydata = np.array( XY1_A57["Force1.Force_x [kNewton]"] )

# Calculate and show fit parameters.
popt, pcov = curve_fit( exp_fit, xdata[0:], ydata[0:] , p0=[100,100,1,-25.],maxfev=1000000)
#popt, pcov = curve_fit( exp_fit, xdata[2:], ydata[2:] ,...


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
Function form:
$$
f(x; k, \beta, \theta) = (\frac{x}{ \theta } )^k \exp{ \left( - \left( \frac{x}{\theta} \right)^{\beta} \right) }
$$
Can you write Python code for it?

def gamma_func_fit(x, k, theta, beta  ):
    return (x/theta)**k * np.exp( - (x/theta)**beta )

# Calculate and show fit parameters. Use a starting guess of p1=1 and p2=0.2
popt, pcov = curve_fit(gamma_func_fit, xdata[3:], ydata[3:] , maxfev=100000)
print(popt)
print(pcov)

# Plot fitted curve along with data
curvex=np.linspace(-1,150,200)
curvey=gamma_func_fit(curvex, *popt  )

plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')
plt.xlabel('xdata')
plt.ylabel('ydata')

plt.show()


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
So there's no data for $h= 0.014 m$.  Same with $h=0.018 m$
Can you write Python code for it?

h = .018 # data.csv, in meters (m) (EY : ???)

ydata = np.array(df.drag[df.h == h])
xdata = np.array(df.v[df.h == h])
print (len(ydata)); print(len(xdata))


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
Some remarks:

Trying different initial conditions for curve_fit helps to find the parameters; wrong choice of initial conditions lead to RunTime Errors since number of iterations run out before residuals shrink enough
I determined the fitting forms by the following manner:
I use only combinations and factors of transcendental, polynomial, exponential, and logarithmic functions, because physical systems should only exhibit such functions
Take a look manually at what the desired fit shape would be and search for what combination of transcendentals, polynomials, exponentials, and logarithms, would give us such as shape
modify accordingly
Can you write Python code for it?

## [`example_ski_data.csv`](https://github.com/capsulecorplab/CurveFit/blob/master/example_ski_data.csv)

# Read csv and save to a pandas data frame
df = pd.read_csv(SUBDIR+'example_ski_data.csv')

df.describe()


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
Taking a look at the plot of drag vs. $v$, the form $p_1 \cos{(p_2 x)} + p_2 \sin{(p_1 x) }$ isn't appropriate since the coefficients in front of the sinusoidal terms $\cos$, $\sin$ are dependent upon the "wavenumbers" $p_2,p_1$.
Instead, consider a gamma function distribution
$$
f(x,k,\theta) = A_0 x^k \exp{ (-\frac{x}{\theta }) }
$$
with $x,k,\theta \in \mathbb{R}$, and $A_0 \in \mathbb{R}$ constant.
Can you write Python code for it?

#def gamma_func_fit(x, k, theta, x_0, A_0 ):
#    return A_0 * x**2 * np.exp( -((x-x_0)**2/theta) )
#def gamma_func_fit(x, k, theta, x_0, A_0 ):
#    return A_0 * x**k * np.exp( -((x-x_0)**2/theta) )
def gamma_func_fit(x, k, theta, x_0  ):
    return x**k * np.exp( -((x-x_0)/theta) )

# Calculate and show fit parameters. Use a starting guess of p1=1 and p2=0.2
popt, pcov = curve_fit(gamma_func_fit, xdata, ydata , maxfev=10000000)

popt

pcov

# Plot fitted curve along with data
curvex=np.linspace(-1,150,200)
#curvey=gamma_func_fit(curvex,popt[0],popt[1],popt[2]  )
curvey=gamma_func_fit(curvex, *popt  )

plt.plot(xdata,ydata,'*')
plt.plot(curvex,curvey,'r')
plt.xlabel('xdata')
plt.ylabel('ydata')

plt.show()


I am working on the file "CurveFit/CurveFit.ipynb".
The first task is:
lift vs. $v$
$$
y=y(x) = A+ B\log{ (x/C) }
$$
Can you write Python code for it?

def log_func_fit(x, A, B, C  ):
    return A+B*np.log( x + C)


I am working on the file "Curves.ipynb".
The first task is:
Astroida:
Can you write Python code for it?

def astroida(t,  A=1):#(x(t),y(t)=(A*cos^3(t), A*sin^3(t))

    return (A*np.power(np.cos(t), 3), A*np.power(np.sin(t), 3))

t=np.linspace(0,2*np.pi, 1000)

(x,y)=astroida(t)
plt.plot(x,y, 'g')
plt.title('Astroida')
plt.xlabel('x(t)')
plt.ylabel('y(t)')
plt.axis('equal')# setarea pe 'equal' are ca efect alegerea aceleasi unitati de masura pe Ox si Oy
                 # in caz contrar (cazul implicit) figura este deformata.


I am working on the file "Curves.ipynb".
The first task is:
Curba $r=\sin(n\theta)$ este un trifoi cu $n$ foi. La fel si $r=\cos(n\theta)$, $\theta\in[0,2\pi]$. Experimentati!
Curbe Bézier
Curbele date parametric sau in coordonate polare se pot genera doar daca se da efectiv expresia analitica
a parametrizarii, respectiv a functiei in coordonate polare.
Curbele Bézier sunt curbe definite procedural, adica pornind de la un numar finit de puncte si un parametru $t\in[0,1]$ se genereaza algoritmic un punct pe o curba.
Curbele Bézier s-au nascut in laboratoarele de la Renault si Citroën in incercarea de a genera
capote pentru automobile,  mai deosebite. Ideea de baza consta in designul unui algoritm care sa genereze profilul unei capote
ce imita un poligon de control.
Can you write Python code for it?


I am working on the file "Curves.ipynb".
The first task is:
Deci functia Curba returneaza un tuple 2D, $(x,y)$, in care fiecare coordonata este un array 1D (un vector). Mai precis, daca notam  cu $(X(t), Y(t))$ coordonatele parametrizarii curbei,
$$X(t)=\cos(t)+t\sin(t), \quad Y(t)=\sin(t)-t\cos(t)$$
atunci vectorii $x$  si $y$, coordonate ale tuple-ului returnat sunt:
$$x=[X(t[0]), X(t[1]), \ldots, X(t[N])]$$
$$y=[Y(t[0]), Y(t[1]), \ldots, Y(t[N])]$$
Apeland acum functia cu un numar mai mare de puncte de diviziune obtinem:
Can you write Python code for it?

(x,y)=Curba(0, 3*np.pi, 1500)
plt.plot(x, y)
plt.xlabel('x(t)')
plt.ylabel('y(t)')
plt.axis('equal')


I am working on the file "Curves.ipynb".
The first task is:
Generalizand la  cazul unei curbe definite de un numar
arbitrar de puncte de control, $ {\bf b}{0},  {\bf b}{1}, \ldots,  {\bf b}_{n}$, $n\geq 1$, dupa
$n$ etape a schemei de Casteljau aplicata dupa acelasi
principiu, folosind un parametru $t\in[0,1]$, se obtine un punct
pe curba Bézier corespunzator acestui parametru.
Punctele de control calculate in etapele intermediare se pot
afisa, teoretic, intr-o matrice triunghiulara de puncte.
 Si anume punctele
 de control sunt puncte initiale, date, deci corespund etapei 0 a procedurii
recursive si le adaugam indicele 0 in pozitia
de sus:
 $$
\begin{array}{llllll}
{\bf b}0^0     &  {\bf b}{0}^1         &  {\bf b}{0}^2& \cdots       & {\bf b}_0^{n-1}& {\bf b}^n_0\
{\bf b}_1^0     &   {\bf b}{1}^1         & {\bf b}{1}^2& \cdots       & {\bf b}_1^{n-1}& \
{\bf b}_2^0     &   {\bf b}{2}^1        & {\bf b}{2}^2   & \cdots       &  & \
\vdots          &          \vdots        &\vdots       &        & & \
{\bf b}{n-2}^0 &  {\bf b}{n-2}^1&       {\bf b}{n-2}^2& & & \
{\bf b}{n-1}^0 &  {\bf b}{n-1}^1       & &  & & \
{\bf b}_n^0     &      & & & &  \end{array} $$
Punctele din coloana
$r$ corespund etapei $r$ a schemei recursive, $r=\overline{1,n}$.
Succint,   schema de Casteljau se exprima prin formula $$ {\bf
b}{i}^{r}(t)=(1-t)\,{\bf b}{i}^{r-1}(t)+t\,{\bf b}{i+1}^{r-1}(t),\:
r=\overline{1,n},\: i=\overline{0,n-r}, $$ adica punctul din pozitia $i$ a  etapei $r$
este o combinatie convexa a punctelor $i$ si $i+1$ din
etapa $r-1$:
$$
\begin{array}{lll} {\bf b}^{r-1}_i&\stackrel{1-t}{\rightarrow}&{\bf
b}^{r}_i\
{\bf b}^{r-1}{i+1}&\stackrel{\nearrow}{t}&\end{array}$$
Desi la prima
vedere s-ar parea ca pentru implementarea schemei de
Casteljau avem nevoie de o matrice triunghiulara de puncte,
in realitate este suficient un sir auxiliar de puncte
$({\bf a}_0,{\bf a}_1,\ldots {\bf a}_n)$, in care la fiecare
apel al schemei de Casteljau se copiaza punctele de control
$({\bf b}_0,{\bf b}_1,\ldots {\bf b}_n)$, ${\bf a}_i={\bf b}_i$,
$i=0,1,\ldots n$.
In etapa $1$, de exemplu,...
Can you write Python code for it?

def deCasteljau(t,b): #punctele de control b_0, b_1, ..., b_n, sunt date intr-un array 2D,
                      #de n+1 linii si 2 coloane. Pe linia i avem coordonatele punctului b_i

    a=np.copy(b) # se copiaza array-ul b in array-ul a
    N=a.shape[0] # interogam cat este numarul de linii ale lui a (deci si ale lui b); n=N-1
    for r in range(1,N): # echivalentul in C a lui for(r=1;r<N, r++); deci 1<=r<=n
        for i in range(N-r):# in C for(i=0; i<N-r;i++)
            a[i,:]=(1-t)*a[i,:]+t*a[i+1,:]# punctul i din etapa r este combinatia convexa
                                          # a punctelor i si i+1 din etapa r-1
    return a[0,:]# a[0,:] contine coordonatele pctului de pe curba coresp lui t

def dreptunghiDesen(b):
    cmin=np.zeros(2) # cmin[i] va stoca coordonata i, minima, a punctelor de control
    cmax=np.zeros(2)# cmax[i] va stoca coordonata i, maxima, a punctelor de control
    for i in range(2):
        cmin[i]=np.amin(b[:,i])-0.5
        cmax[i]=np.amax(b[:,i])+0.5
    return (cmin, cmax)

def curbaBezier(b, nr=100):#nr=nr de puncte ce se calculeaza pe curba
    h=1.0/nr
    pcteC=[]
    for k in range(nr):
        t=k*h# 0.01 este pasul de divizare a intervalului [0,1], al parametrului t
        P=deCasteljau(t,b)# P punct pe curba Bezier, corespunzator lui t
        pcteC.append(P)
    return pcteC# functia returneaza lista punctelor de pe curba calculate

def DrawBezier(b):
    cmin, cmax=dreptunghiDesen(b)
    plt.axis([cmin[0], cmax[0], cmin[1], cmax[1]])
    plt.plot(b[:, 0], b[:,1],  'bo',  b[:, 0], b[:,1])#marcheaza punctele de control si
                                                      #traseaza poligonul de control
    pcteC=curbaBezier(b)# pcteC este lista punctelor calculate pe curba

    Xpcte, Ypcte=zip(*pcteC)# functia zip() extrage din lista  pcteC, lista x-silor, si y-cilor

    plt.plot(Xpcte, Ypcte, 'r')

################
# Pentru a a genera o curba Bezier este suficient sa declaram array-ul...


I am working on the file "Curves.ipynb".
The first task is:
Gradul unei curbe Bézier este mai mic cu o unitate
decat numarul punctelor sale de control. Astfel o curba
Bézier de grad 1 este generata de doua puncte de control
${\bf b}_0, {\bf b}_1$, $b(t)={\bf b}_0 B_0^1(t)+{\bf b}_1B_1^1(t)$ $=(1-t){\bf b}_0+t{\bf b}_1=
{\bf b}_0

t\vec{{\bf b}_0{\bf b}_1}$, si este segmentul determinat de cele doua
puncte.

O curba Bézier  generata de trei puncte de control este un arc
de parabola:
Can you write Python code for it?


I am working on the file "Curves.ipynb".
The first task is:
In etapa 1 se
determina pe fiecare segment determinat de doua puncte de
control consecutive, punctul ce imparte segmentul respectiv
in acelasi raport $\displaystyle\frac{t}{1-t}$ in care $t$ imparte pe $[0,1]$:

$$\begin{array}{lll}{\bf b}^1_0&=&(1-t)
{\bf b}_0^0+t{\bf b}^0_1\
{\bf b}^1_1&=&(1-t){\bf b}_1^0+t{\bf b}^0_2\end{array}$$

In etapa a doua, pe
segmentul determinat de cele doua puncte calculate in etapa
precedenta se determina punctul ${\bf b}^2_0$ ce
imparte segmentul $[{\bf b}^1_0, {\bf b}^1_1]$ in raportul
$\displaystyle\frac{t}{1-t}$:

$${\bf b}^2_0=(1-t){\bf
b}^1_0+t{\bf b}^1_1$$
Inlocuind punctele calculate in etapa 1  in exprimarea lui ${\bf b}0^2$, calculat in etapa 2,
 obtinem:
$$\begin{array}{lll}{\bf b}^2_0&=&(1-t)[(1-t){\bf
b}_0^0+t{\bf b}^0_1]+t[(1-t){\bf b}_1^0+t{\bf
b}^0_2]=\&=&\underbrace{(1-t)^2}{B^2_0(t)}{\bf
b}0^0+\underbrace{2t(1-t)}{B^2_1(t)}{\bf b}1^0+\underbrace{t^2}{B^2_2(t)}{\bf
b}^0_2=b(t)\end{array},$$  adica punctul $b^2_0$ reprezinta parametrizarea curbei data de Bézier, in baza
Bernstein, evaluata in parametrul $t$.
Remarcam ca  in fiecare etapa a schemei de
Casteljau, numarul punctelor calculate se reduce cu o unitate
fata de etapa precedenta si in ultima etapa
rezulta punctul de pe curba Bézier, corespunzator parametrului
$t$.
In figura urmatoare ilustram algoritmul de Casteljau pentru o curba definita de 4 puncte de control:
Can you write Python code for it?


I am working on the file "Curves.ipynb".
The first task is:
In figura de mai sus multimea din stanga este convexa, iar cea din dreapta neconvexa, pt ca desi punctele
marcate cu albastru apartin multimii, segmentul ce le uneste nu este in intregime inclus in multime.
O combinatie convexa a punctelor
 $A_0, A_1, \ldots, A_n$  din $\mathbb{R}^2$ sau $\mathbb{R}^3$ este de forma:
 $$
\alpha_0A_0+\alpha_1A_1+\cdots+\alpha_n A_n, \quad \mbox{unde}\,\, \alpha_i\in [0,1],\: \mbox{si}\,\, \alpha_0+\alpha_1+\cdots+\alpha_n=1.$$
Sa aratam ca o combinatie convexa reprezinta un punct. Intr-adevar, din relatia
 $\alpha_0+\alpha_1+\cdots+\alpha_n=1$ exprimam pe $\alpha_0=1-(\alpha_1+\alpha_2+\cdots+\alpha_n)$,  il inlocuim in relatia ce defineste combinatia convexa si obtinem:
$$\begin{array}{l}A_0+\alpha_1(A_1-A_0)+\alpha_2(A_2-A_0)+\cdots+\alpha_n(A_n-A_0)=\
\underbrace{A_0}{punct}+\underbrace{\alpha_1\overrightarrow{A_0A_1}+\alpha_2\overrightarrow{A_0A_2}+\cdots+\alpha_n\overrightarrow{A_0A_n}}{vector}:=\underbrace{P}_{punct}\end{array}$$
Infasuratoare convexa a punctelor $A_0, A_1, \ldots, A_n$ este multimea  tuturor combinatiilor convexe
ale acestor puncte.

Infasuratoarea convexa a doua puncte, $A, B$, este segmentul de extremitati
$A, B$, adica $[A,B]\stackrel{def}{=}{P\,|\, P=A+s\vec{AB},
s\in[0,1]}$.

Infasuratoarea a trei puncte necoliniare, $A_0, A_1, A_2$ este compactul triunghiular $\triangle{A_0A_1A_2}$.
Can you write Python code for it?


I am working on the file "Curves.ipynb".
The first task is:
In ultimul bloc de cod puteti inlocui pe $t$ cu diverse valori in $[0,1]$ si vedeti directia vitezei.
Curbele Bezier se genereaza deobicei interactiv, alegand punctele de control cu mouse-ul si apoi
trasand curba corespunzatoare.
Pentru generarea interactiva veti primi un script ce se ruleaza in Spyder.
Can you write Python code for it?

from IPython.core.display import HTML
def  css_styling():
    styles = open("./custom.css", "r").read()
    return HTML(styles)
css_styling()


I am working on the file "Curves.ipynb".
The first task is:
Infasuratoarea convexa a $n$ puncte din plan este cel mai mic poligon convex ce contine toate punctele.
In figura de mai sus infasuratoarea convexa  apunctelor $C_0, C_1, C_2, C_3, C_4$ este
poligonul plin, de varfuri $C_0, C_1, C_4, C_2$.
Daca $A, B$ sunt doua puncte in
plan sau spatiu, o combinatie convexa a lor este un
punct $M=\alpha_1 A+\alpha_2 B$, $\alpha_i\in[0,1]$, $\alpha_1+\alpha_2=1$.
Notand $\alpha_2=t$, rezulta ca $\alpha_1=1-t$ si deci
punctul $M$ se exprima ca o combinatie convexa a lui $A$
si $B$ astfel:
$$M=(1-t)A+tB, \,\, t\in[0,1]$$
Punctul $M$ astfel definit apartine segmentului $[A,B]$.
Daca $\vec{AM}=r\vec{MB}$, spunem ca punctul $M\neq B$, imparte segmentul
$[A,B]$ in raportul $r$.
Un
punct $M\neq B$ al segmentului $[A,B]$, $M=(1-t)A+tB$, imparte
segmentul in raportul $\displaystyle\frac{t}{1-t}$ si reciproc,
daca $M$ imparte segmentul $[A,B]$ in raportul
$\displaystyle\frac{t}{1-t}$, $t\neq 1$, atunci $M=(1-t)A+tB$.
Intr-adevar:
$$\begin{array}{l}M=(1-t)A+tB\,\, \Leftrightarrow\,\,
M=A+t\vec{AB}\,\,\Leftrightarrow\,\, \vec{AM}=t(\vec{AM}+\vec{MB})\,\,\Leftrightarrow\
\Leftrightarrow\,\, (1-t)\vec{AM}=t\vec{MB}\,\,\Leftrightarrow
\vec{AM}=\displaystyle\frac{t}{1-t}\vec{MB}\end{array}$$
Pornind de la aceasta proprietate explicam mai
intai schema lui de Casteljau pentru cazul curbei
definita de 3 puncte de control ${\bf b}_0, {\bf b}_1, {\bf b}_2$. Acestea fiind
punctele initiale (date), se renoteaza ${\bf b}_0^0, {\bf b}^0_1, {\bf
b}^0_2$, indicele $0$ din pozitia de sus indicand etapa 0 a
procedurii.
Fixand un parametru $t\in[0,1)$, acesta
imparte segmentul $[0,1]$ in raportul $\displaystyle\frac{t}{1-t}$.
Can you write Python code for it?


I am working on the file "Curves.ipynb".
The first task is:
O curba Bézier interpoleaza
extremitatile poligonului sau de control (adica trece sigur prin ${\bf b}_0$ si ${\bf b}_n$,
deoarece daca $b$ este parametrizarea Bernstein, atunci $b(0)={\bf b}_0$ si $b(1)={\bf b}_n$. O curba Bézier nu  trece insa  prin punctele de control intermediare.

O curba Bézier imita forma poligonului
de control.
Can you write Python code for it?


I am working on the file "Curves.ipynb".
The first task is:
Pentru a genera vectorii viteza/acceleratie in cateva momente, $t_i$, asociati traiectoriei parametrizate de $r(t)=(x(t), y(t))$, se evalueaza vectorul $\dot{\vec{r}}(t)$, respectiv $\ddot{\vec{r}}(t)$
in $t_i$ si se apeleaza functia plt.quiver(X, Y, U, V) unde  $X$, respectiv $Y$,  este array-ul 1D de coordonate
$x(t_i)$, respectiv  $y(t_i)$ , iar $U$ si $V$ au respectiv coordonatele $x'(t_i)$ si $y'(t_i)$,
in cazul vitezei, si $x''(t_i)$, $y''(t_i)$, in cazul acceleratiei.
De exemplu sa trasam curba parametrizata de:
$$x(t)=t\sin(t), \quad y(t)=\cos(t), \quad t\in[0, 2\pi]$$
si vectorii viteza, respectiv acceleratie, in 8 puncte ale traiectoriei, atinse in miscarea punctului mobil in  momentele $T[i]$, unde T=np.linspace(0,2*np.pi, 8):
Can you write Python code for it?

def r(t):
    return(t*np.sin(t), 1.5*np.cos(t))

def rprim(t):
    return (np.sin(t)+t*np.cos(t), -1.5*np.sin(t))

def rsecund(t):
    return (2*np.cos(t)-t*np.sin(t), -1.5*np.cos(t))

plt.axis([-5,2, -1.75, 1.75])
t=np.arange(0, 2*np.pi, 0.01)
x,y=r(t)
plt.plot(x,y)#traseaza curba

T=np.linspace(0,2*np.pi, 10)
X,Y=r(T)
U,V=rprim(T)
plt.quiver(X,Y,U,V, color='r', units='x', scale=5)
normv=np.sqrt(U**2+V**2)
print 'normele vectorilor viteza:\n', normv.round(3)
W,Z=rsecund(T)
plt.quiver(X,Y,W,Z, color='g', units='x', scale=5)
norma=np.sqrt(W**2+Z**2)
print 'normele vectorilor acceleratie:\n', norma.round(3)


I am working on the file "Curves.ipynb".
The first task is:
Pentru a intelege definitia acestei functii, o apelam mai intai pentru un $N$ mic si afisam tipul
coordonatelor tuple-ului returnat:
Can you write Python code for it?

(x,y)=Curba(0, 3*np.pi, 15)
plt.plot(x, y)
plt.axis('equal')

print type(x), type(y)

print x.round(3)

print y.round(3)