I am working on the file "SC1v2/Bonus - 00 - The beginning.ipynb".
The first task is:
e) Broadcasting
Can you write Python code for it?

# Creating a 1D array, this is the stringly suggested way to create "vectors" is NumPy.
# This can be considered as a row vector.
x1 = np.array([1,2,3])
print(textBoldH('x1:'),x1)
print(textBoldH('shape:'),x1.shape)
print(textBoldH('dtype:'),x1.dtype)
print(textBoldH('type name:'),type(x1).__name__)

# Creating a 2D array with one row.
x2 = x1[np.newaxis] # What does np.newaxis do?
print(textBoldH('x2:'),x2)
print(textBoldH('shape:'),x2.shape)
print(textBoldH('dtype:'),x2.dtype)
print(textBoldH('type name:'),type(x2).__name__)
# This emulates a column vector.

# Transpossing the 2D array. This emulates a column vector.
x2 = x1[np.newaxis].T
print(textBoldH('x2:'),x2)
print(textBoldH('shape:'),x2.shape)
print(textBoldH('dtype:'),x2.dtype)
print(textBoldH('type name:'),type(x2).__name__)

# Notice that transpossing a 1D array does not do anything (please try it by yourself!).
# Now that we have a 1D array (that emulates a row vector) and
# a 1 column 2d array (that emulates a columns vectors), we can do a "BROADCAST" and see what happend.
x3=x1+x2
print(textBoldH('x3:'),x3)
print(textBoldH('shape:'),x3.shape)
print(textBoldH('dtype:'),x3.dtype)
print(textBoldH('type name:'),type(x3).__name__)
# In this cas we have operated elementwise and broadcasted the array.

# The previous operation is equivalent to the following operation.
x11 = np.tile(x1, (3,1)) # Repeat the 1d array 'vertically' 3 times
print(textBoldH('x11:'),x11)
x21 = np.tile(x2, 3) # Repeat the 2d array 'horizontally' 3 times
print(textBoldH('x21:'),x21)
x31 = x11+x21
print(textBoldH('x31:'),x31)
print(textBoldH('shape:'),x31.shape)
print(textBoldH('dtype:'),x31.dtype)
print(textBoldH('type name:'),type(x31).__name__)
print(textBoldR(' As you may suspect, this way of doing the sum is not recommended.\n'+
               ' It was added just for comparison purposes.'))


I am working on the file "SC1v2/Bonus - 00 - The beginning.ipynb".
The first task is:
f) Reduction
Can you write Python code for it?

print(textBoldH('a3\n'), a3)

print('\n')
print(textBoldB(' First reduction, sum along axis 0:'))
print(textBoldB(' s1 = np.sum(a3,axis=0)'))
s1 = np.sum(a3,axis=0)
print(textBoldH('s1'), s1)
print(textBoldH('shape:'),s1.shape)
print(textBoldH('dtype:'),s1.dtype)
print(textBoldH('type name:'),type(s1).__name__)

print('\n')
print(textBoldB(' First reduction, sum along axis 1:'))
print(textBoldB(' s2 = np.sum(a3,axis=1)'))
s2 = np.sum(a3,axis=1)
print(textBoldH('s2'), s2)
print(textBoldH('shape:'),s2.shape)
print(textBoldH('dtype:'),s2.dtype)
print(textBoldH('type name:'),type(s2).__name__)

print('\n')
print(textBoldB(' First reduction, sum along axis 0 and 1:'))
print(textBoldB(' s3 = np.sum(a3)'))
s3 = np.sum(a3)
print(textBoldH('s3'), s3)
print(textBoldH('shape:'),s3.shape)
print(textBoldH('dtype:'),s3.dtype)
print(textBoldH('type name:'),type(s3).__name__)

print('\n')
print(textBoldR(' As you may suspect, the reduction reduces the dimensionality in respect of the original input.'))


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
As the plot shows, the solution is negative but tends to $0$ from the negative numbers.
To visualize a bit better we will use a log-scale and plot the absolute value of the solution, since we can't compute the logarithm of negative numbers.
Can you write Python code for it?

b = np.linspace(2,1000,1000) # Just using a higher resolution for plotting
plt.figure()
plt.semilogy(b,np.abs(x_pos(b)),'r-',label='$|x_{+}(b)|$')
plt.grid(True)
plt.xlabel('$b$')
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
Conclusion
There is not an algorithm to solve these kind of problems, they need to by analyzed carefully to point out where you may have an issue.
In particular, a good place to look is when you need to substract numbers, which is actually what happended here, but it is not always the case.
As we say in the other jupyter notebook (using Julia!) to compute $\sum_{k=1}^\infty \frac{1}{k^2}$.
The good news is that you are aware of the problem now!
Quick questions:
- Will the negative root have a similar problem?
- Can you define a general solver for a quadratic equation that always work?
<div id='acknowledgements' />

Acknowledgements
Back to TOC

Material originally created by professor Claudio Torres (ctorres@inf.utfsm.cl). v.1.0. DI UTFSM. March 2022._
Can you write Python code for it?

# Extra example, the following code show the log2 of the "machine epsilon" used by Python for the indicated data types.
# Isn't it interesting? Does not seem to be standard?
# see: https://numpy.org/doc/stable/user/basics.types.html
print(np.log2(np.finfo(np.float64).eps))
print(np.log2(np.finfo(np.longdouble).eps))
print(np.log2(np.finfo(np.float128).eps))


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
For compeltenes, we will show the evolution of the root as a function of $b$, the same plot showed few cells back.
Can you write Python code for it?

b = np.logspace(np.log10(2),11,1000) # Just using a higher resolution for plotting
plt.figure()
plt.loglog(b,np.abs(x_pos(b)),'r-',label='$|x_{+}(b)|$')
plt.loglog(b,np.abs(x_pos_fixed(b)),'b-',label='$|x_{+}(b)|$ (fixed)')
plt.grid(True)
plt.xlabel('$b$')
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
Here we show the magnitude of the difference between the roots, it show an increasing behaviour that seems to improve later.
This is actually a tricky plot.
First, the scale on the $y$-axis is small.
Second, when it shows a better behaviour, is because the fixed root is actually going to zero an the original root is zero, which is wrong.
So please don't get confuse and conclude that thing gets better as $b$ increases!
It is actually totally the opposite!
Can you write Python code for it?

b = np.logspace(np.log10(2),11,1000) # Just using a higher resolution for plotting
plt.figure()
plt.loglog(b,np.abs(x_pos(b)-x_pos_fixed(b)),'.k',label='$|x_{+}(b)-x_{+}(b)$ (fixed)$|$')
plt.grid(True)
plt.xlabel('$b$')
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
Now, in this plot we observe that the classic quadratic formula behaves well for the values of $b$ in the range $[2,10^7]$ approximately.
And we clearly observe that for values of approximately $b=10^8$ there seems to be a problem.
Even for greater values the solutions does not show up.
Actually, what it is happening is that for large values, the solution it provides the quadratic formula is $0$, but since we are in log-log scales, it does not show up because it would be $-\infty$.
To observe this, we will add a line with the value $10^{-20}$ to be able to see something.
Can you write Python code for it?

b = np.logspace(np.log10(2),11,1000) # Just using a higher resolution for plotting
plt.figure()
plt.loglog(b,np.abs(x_pos(b)),'r-',label='$|x_{+}(b)|$')
plt.loglog(b,b*0+np.power(10.,-20),'--',label='$10^{-20}$',alpha=0.7)
plt.grid(True)
plt.xlabel('$b$')
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
So, as you may notice from before, we don't see any unexpected behavior for now.
The well-known quadratic formula works just fine.
We can even plot the outcome and used a vectorized approach.
Can you write Python code for it?

b = np.linspace(2,1000,1000) # Just using a higher resolution for plotting
plt.figure()
plt.plot(b,x_pos(b),'-',label='$x_{+}(b)$')
plt.grid(True)
plt.xlabel('$b$')
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
The previous outcome proves that the value obtained is actually $0$.
So, the following questions seem critical,
1. What is happening here?
2. Why is it only happening for large values of $b$?
3. Can this be fixed?
See next section for the answers.
A more detailed analysis
As we have observed numerically, there is an issue with the direct evaluation of the positive root of the quadratic formula for large values of $b$.
Note that the issue is not always with the positive root, this is just an toy example. The negative root may also show the problem if the same conditions appeared.
For completeness we repeat the formula here:
$$
\begin{equation}
    x_{+} = \dfrac{-b+\sqrt{b^2-4}}{2}.
\end{equation}
$$
To answer the first question we will reproduce algebraically what it is happening for the case $b=10^9$:
$$
\begin{align}
    x_{+}   &= \dfrac{-b+\sqrt{b^2-4}}{2}\
            &= \dfrac{-10^9+\sqrt{(10^9)^2-4}}{2}\
            &= \dfrac{-10^9+\sqrt{10^{18}-4}}{2}.
\end{align}
$$
So, here is the first numercal challenge, can we actually perform the computation of $10^{18}-4$.
To understand this, we will show the floating point representation of each number and the result of the operation.
Can you write Python code for it?

print(textBoldH("Floating Point representation of the Double Precision (FPDP) of number $10^{18}$"))
to_fps_double(1.e18)
print(textBoldH("FPDP of number $4$"))
to_fps_double(4.)
print(textBoldH("FPDP of number $10^{18}-4$"))
to_fps_double((1.e18)-4.)
print(textBoldH("Just to make sure, we compute $(10^{18}-4)-10^{18}$"))
to_fps_double(((1.e18)-4.)-1.e18)
print(textBoldR("So the outcome is indeed $0$, so substracting 4 does not do anything"))


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
This partially answer the question, now, we need to compute the square root.
Can you write Python code for it?

print(textBoldH("FPDP of number $\sqrt{10^{18}}$"))
to_fps_double(np.sqrt(1.e18))
print(textBoldH("FPDP of number $10^9$"))
to_fps_double(1.e9)


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
Thus, we have sucessfully confirmed that the formula is indeed correct.
The idea os using SymPy is that it can help us to manipulate complex algebraic expressions, for instance, we will reproduce now two classical facts derived from the quadratic formula.
- $x_{-}\,x_{+}$
- $x_{-}+x_{+}$
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
To push it to the limit, we will use a log-log scale, i.e. we will chose $b$ in a log scale, basically increasing its value.
Can you write Python code for it?

b = np.logspace(np.log10(2),11,1000) # Just using a higher resolution for plotting
plt.figure()
plt.loglog(b,np.abs(x_pos(b)),'r-',label='$|x_{+}(b)|$')
plt.grid(True)
plt.xlabel('$b$')
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 02 - Quadratic formula.ipynb".
The first task is:
What we observe from the previous numerical experiment is that the $\sqrt{10^{18}}$ is exactly $10^9$, which is indeed the expected result.
Now, what does this imply?
Let's put the numerical computation in an algebraic form.
We will be adding fl $(\cdot)$ where we perform the corresponding numerical computation:
$$
\begin{align}
    \text{fl}(x_{+})
        &= \dfrac{-10^9+\sqrt{\text{fl}(10^{18}-4)}}{2}\
        &= \dfrac{-10^9+\sqrt{\text{fl}(10^{18})}}{2}\
        &= \dfrac{-10^9+\text{fl}(\sqrt{10^{18}})}{2}\
        &= \dfrac{\text{fl}(-10^9+10^9)}{2}\
        &= 0.
\end{align}
$$
Thus, this analysis answer teh first and second questions, which we will repeat here for completeness:
1. What is happening here? Answer: There is loss of significance.
2. Why is it only happening for large values of $b$? Answer: This only happens for large values of $b$ because that is the loss of significance occurs in this case. Particularly, it happens when there is a sum (or substraction) of number that have a difference of more than 16-order of magnitude.
Fixing the issue
To answer the third question, we need to think in a way such that we avoid forcing the code to operate between numbers that have a large difference of magnitud in critical steps.
In this case, we can multiply by a convenient $1$ to help us to cancel out the issue.
This means:
$$
\begin{align}
    x_{+}   &= \dfrac{-b+\sqrt{b^2-4}}{2}\
            &= \dfrac{-b+\sqrt{b^2-4}}{2} \,\dfrac{(-b-\sqrt{b^2-4})}{(-b-\sqrt{b^2-4})}\
            &= \dfrac{(-b)^2-(b^2-4)}{2\,(-\,b-\sqrt{b^2-4})}\
            &= \dfrac{-2}{b+\sqrt{b^2-4}}.
\end{align}
$$
Notice that in the last step we have conveniently manipulated the mathematical expression simplifying it as much as possible.
Then, we can implement this new expression now and solve the third question!
Can you write Python code for it?

def x_pos_fixed(b):
    return -2./(b+np.sqrt(np.power(b,2)-4.))


I am working on the file "SC1v2/Bonus - 03 - An slow root finder.ipynb".
The first task is:
<div id='FNE' />

The First Numerical Experiment
Back to TOC
The output looks similar to the output generated by the Bisection Method, however in this case we added the index for the internal loop that goes over each interval with the index $i$.
Can you write Python code for it?

r, m = ASRF(a=0, b=1, f=f2, N=5)
print(r, m)


I am working on the file "SC1v2/Bonus - 03 - An slow root finder.ipynb".
The first task is:
<div id='SNE' />

The Second Numerical Experiment
Back to TOC
This second numerical experiments computed the number of function evaluation required to obtain the root of $f_2(x)$ as we change the number of intervals used.
Notice that when $N=2$ you may think we should actually get back the bisection method, but we, unfortunately, don't.
In particular, it seems that using $N=8$ we get the best performance, which is $56$ function evaluations.
Can you write Python code for it?

data_number_of_function_evaluations = []
# The range of intervals to be used
Ns = np.arange(2,21)
for N in Ns:
    r, m = ASRF(a=0, b=1, f=f2, N=N, flag_OUTPUT=False)
    data_number_of_function_evaluations.append(m)
plt.figure(figsize=(10,10))
plt.plot(Ns,data_number_of_function_evaluations,'.', markersize=10)
plt.title('ASRF')
plt.xticks(Ns)
plt.xlabel('# of intervals N')
plt.ylabel('# of function evaluations')
plt.grid(True)
plt.ylim([0, 110])
plt.show()


I am working on the file "SC1v2/Bonus - 03 - An slow root finder.ipynb".
The first task is:
<div id='TNE' />

The Third Numerical Experiment
Back to TOC
This last numerical experiment is the largest case, we test the ASRF method against the Bisection method.
We select the range from $2$ up to $20$ intervals and we selected $12$ roots equalspaced from $h=0.12$ up to $0.92$,
this may seem like arbitrary numbers and they are, so we invite you to try with different number, but just remember that if you change the range, you must change the values for $a$ and $b$ accordingly.
Can you write Python code for it?

# Range of number of intervals used, starting from 2.
Ns = np.arange(2,21)
# Range of roots tested.
Hs = np.linspace(0.12,0.92,12)

# Running ASRF numerical experiments.
all_output_ASRF = []
for h in Hs:
    print('Running numerical experiment with h=',h)
    output_ASRF= []
    for N in Ns:
        rN, m = ASRF(a=0, b=1, f=lambda x: f(x,h), N=N, flag_OUTPUT=False)
        output_ASRF.append(m)
    all_output_ASRF.append(output_ASRF)

# Running Bisection method experiments.
out_bisection = []
for h in Hs:
    r, m = bisect(f=lambda x: f(x,h), a=0, b=1,flag_OUTPUT=False)
    out_bisection.append(m)


I am working on the file "SC1v2/Bonus - 03 - An slow root finder.ipynb".
The first task is:
<div id='intro' />

Introduction
Back to TOC
In this jupyter notebook we will study the behaviour of a ABSRF (A Bad and Slow Root Finder).
Notice that we have added "(?)" in the title since we still need to evaluate it in kore detail, basically, we will make a comparison with the bisection method itself.
We will study this root finder because it has been traditionally proposed every time this class has been taught and it is important to quantify its behaviour, this means, to understand how fast it converges to a root and how long it takes.
The algorithm is as follows:
- Consider we have a continuous function $f(x)$ that has a root in the interval $[a,b]$.
- The ABSRF divides the interval in $N$ equalspaced segments and then it finds in which interval there is a change of sign, it then subdivides that interval and continues the procedure.
The algorithm is simple and it will be interesting to analyze it.
Can you write Python code for it?

'''
INPUT:
a   : (double)  Left end of interval [a,b] where we will find the root.
b   : (double)  Right end of interval [a,b] where we will find the root.
f   : (lambda)  Lambda function that we will find an approximation of a root.
N   : (integer) Number of segments we will be using in each decomposition of the interval.
TOL : (double)  Tolerance for converge we will be using.
flag_OUTPUT : (bool) Flag to show the evolution of the algorithm.
M   : (int) Max number of evaluations allowed.

OUTPUT:
r   : (double)  The approximation found for the root.
m   : (int)     Number of functions evaluations computed.
'''
def ASRF(a,b,f,N=10,TOL=1e-8, flag_OUTPUT=True, M=int(1e2)):
    fa = f(a)
    m  = 1 # Counting the number of evaluations used.
    r  = a
    computed_error = 2*TOL+1
    # Output table to store the numerical evolution of the algorithm
    output_table = []

    # If the root is at "a"
    if np.abs((b-a)/2.)<TOL:
        r = a
        return r, m
    # If there is no change of sign
    m = m+1
    if fa*f(b)>=0:
        print('Error: No change of sign')
        return np.nan, m

   # N must be at least 3
    if N<2:
        print('Error: N must be at least 2')
        return np.nan, m

    # Counter for the number of lines of output.
    k=0
    # Iterate until the tolerance is reached
    while computed_error > TOL:
        X = np.linspace(a,b,N+1)
        for i, xi in np.ndenumerate(X[1:]):
            # There is key difference respecto to the bisection method at this point,
            # the difference is that in general we need to determine in which interval
            # it is the root, where in the bisection method we know that if it is not
            # in one interval, it must be on the other interval. Of course we need
            # to consider the case when the root is at $x=c$.
            m = m+1
            fxi=f(xi)
            if fa*fxi<0:
                a = X[i]
                b = xi
                r = (a+b)/2.
                fa = f(a)
                m = m+1
     ...


I am working on the file "SC1v2/Bonus - 03 - An slow root finder.ipynb".
The first task is:
The following code is just the implementation of the Bisection method.
We include it here just for comparison purposes.
Can you write Python code for it?

# This implemenatation of the bisection method was obtained from the
# Jupyter Notebook called '03_roots_of_1D_equations.ipynb', the main difference
# is that in this implementation returns the number of function
# evaluations used.
def bisect(f, a, b, tol=1e-8, maxNumberIterations=100,flag_OUTPUT=True):
    # Evaluating the extreme points of the interval provided
    fa = f(a)
    m = 1 # This variable will be used to count the number of function evaluations
    fb = f(b)
    m = m+1
    # Iteration counter.
    i = 0
    # Just checking if the sign is not negative => not root  necessarily
    if np.sign(fa*fb) >= 0: # This was updated!
        print('f(a)f(b)<0 not satisfied!')
        return None

    # Output table to store the numerical evolution of the algorithm
    output_table = []

    # Main loop: it will iterate until it satisfies one of the two criterias:
    # The tolerance 'tol' is achived or the max number of iterations is reached.
    while ((b-a)/2 > tol) and i<=maxNumberIterations:
        # Obtaining the midpoint of the interval. Quick question: What could happen if a different point is used?
        c = (a+b)/2.
        # Evaluating the mid point
        fc = f(c)
        m = m+1
        # Saving the output data
        output_table.append([i, a, c, b, fa, fc, fb, b-a])

        # Did we find the root?
        if fc == 0:
            print('f(c)==0')
            break
        elif np.sign(fa*fc) < 0:
            # This first case consider that the new inetrval is defined by [a,c]
            b = c
            fb = fc
        else:
            # This second case consider that the new interval is defined by [c,b]
            a = c
            fa = fc
        # Increasing the iteration counter
        i += 1
    if flag_OUTPUT:
        # Showing final output table
        columns    = ['$i$', '$a_i$', '$c_i$', '$b_i$', '$f(a_i)$', '$f(c_i)$', '$f(b_i)$', '$b_i-a_i$']
        df = pd.DataFrame(data=output_table, columns=columns)
        display(df)

    # Computing the best...


I am working on the file "SC1v2/Bonus - 04 - Newton Rn.ipynb".
The first task is:
Here we implement the Newton's method in higher dimension in a widget.
Most of the code is for plotting and showing the outcome, but the core, i.e. the Newton steps, is just 1 line of code!
Can you write Python code for it?

def Show_Newton(x0=1.2,y0=0.3,n=0):
    plt.figure()
    CS1 = plt.contour(X, Y, Z1,levels=[0])
    CS2 = plt.contour(X, Y, Z2,levels=[0])
    plt.grid()
    plt.axis('equal')
    plt.title(r'Newton $\mathbb{R}^n$')
    plt.plot(x0,y0,'rx')
    x_previous = np.array([x0,y0])
    print('Initial guess: [%.10f, %.10f]' % (x0,y0))
    for i in np.arange(n):
        x_next=NewtonStep(x_previous)
        x1,y1 = x_next
        plt.plot(x1,y1,'rx')
        plt.plot([x0, x1],[y0, y1],'r')
        x0=x1
        y0=y1
        x_previous = x_next
        print('Iteration %d  : [%.10f, %.10f]' % (i+1,x1,y1))
    plt.show()

interact(Show_Newton,x0=(-1.4,1.4,0.1),y0=(-1.4,1.4,0.1), n=(0,100,1))


I am working on the file "SC1v2/Bonus - 04 - Newton Rn.ipynb".
The first task is:
The Newton Step takes advantage of the vectorized implementation!
Can you write Python code for it?

# Here we use 'np.linalg.solve', but we could use LU or PALU! Or iterative methods!
NewtonStep = lambda xi: xi-np.linalg.solve(J(xi),F(xi))


I am working on the file "SC1v2/Bonus - 05 - Finding 2 Chebyshev PointsGraphically.ipynb".
The first task is:
Finally, we have included the min in the plot and we see the agreement between the min of $\omega(x_1,x_2)$ and the Chebyshev points found.
Can you write Python code for it?

plt.figure(figsize=(8,8))
plt.contourf(X, Y, W,100, cmap=cm.nipy_spectral, antialiased=False)
plt.plot(c1,c2,'w.',markersize=16)
plt.plot(c2,c1,'w.',markersize=16)
plt.colorbar()
plt.xlabel(r'$\widehat{x}_1$')
plt.ylabel(r'$\widehat{x}_2$')
plt.show()


I am working on the file "SC1v2/Bonus - 05 - Finding 2 Chebyshev PointsGraphically.ipynb".
The first task is:
For instance, if you want to evaluate at $\widehat{x}_1=0.2$ and $\widehat{x}_2=0.5$ you get,
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 05 - Finding 2 Chebyshev PointsGraphically.ipynb".
The first task is:
Recall that the Chebyshev points are points that minimize the following expression:
$$
\displaystyle{\omega(x_1,x_2,\dots,x_n)=\max_{x} |(x-x_1)\,(x-x_2)\,\cdots\,(x-x_n)|}.
$$
This comes from the Interpolation Error Formula (I hope you remember it, otherwise see the textbook or the classnotes!).
In this notebook, we will find the $\min$ for 2 points,
this means:
$$
\begin{align}
[x_1,x_2]&= \displaystyle{\mathop{\mathrm{argmin}}{\widehat{x}_1,\widehat{x}_2\in [-1,1]}} \,\omega(\widehat{x}_1,\widehat{x}_2)\
&=\displaystyle{\mathop{\mathrm{argmin}}{\widehat{x}1,\widehat{x}_2\in [-1,1]}}\,
\max{x\in [-1,1]} |(x-\widehat{x}_1)\,(x-\widehat{x}_2)|.
\end{align}
$$
For doing this, we first need to build $\omega(\widehat{x}_1,\widehat{x}_2)$,
Can you write Python code for it?

N=50
x=np.linspace(-1,1,N)
w = lambda x1,x2: np.max(np.abs((x-x1)*(x-x2)))
wv=np.vectorize(w)


I am working on the file "SC1v2/Bonus - 05 - Finding 2 Chebyshev PointsGraphically.ipynb".
The first task is:
Thus, the idea is to find the minimim.
Now we need to evaluate $\omega(x_1,x_2)$ over the domain $\Omega=[0,1]^2$.
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 05 - Finding 2 Chebyshev PointsGraphically.ipynb".
The first task is:
With this data, we can now plot the function $\omega(x_1,x_2)$ on $\Omega$.
The minimun value of is shown by the color at the bottom of the colorbar.
By visual inspection, we see that we have two mins.
They are located at the bottom right and top left.
Can you write Python code for it?

plt.figure(figsize=(8,8))
#plt.contourf(X, Y, W,100, cmap=cm.hsv, antialiased=False)
plt.contourf(X, Y, W,100, cmap=cm.nipy_spectral, antialiased=False)
plt.xlabel(r'$\widehat{x}_1$')
plt.ylabel(r'$\widehat{x}_2$')
plt.colorbar()
plt.show()


I am working on the file "SC1v2/Bonus - 05 - Newton's divided differences, Sinc and piecewiselinear interpolations.ipynb".
The first task is:
<div id='AnalysisTimeVsError' />

Analysis of time vs error for interpolation algorithms
Back to TOC
The idea here is to understand the relationship between the error we get for each algorithm and the computation time associated.
This analysis highlights the advantage of Barycentric interpolation with Chebyshev points when the function being interpolated is analytic.
Notice that the first plot shows the computation time required for the construction of the interpolant, the second the cost of evaluation of the interpolant in a finer grid and the third one shows the sum of both times.
Can you write Python code for it?

def get_error_and_time(f, xmin=-1, xmax=1, method=Lagrange, points=np.linspace):

    construction_times   = []
    evaluation_times     = []
    interpolation_errors = []

    #n_range = np.logspace(1, 2, num=5, endpoint=True, base=10, dtype=int)
    n_range = np.array([2,4,8,16,30,50])

    for n in n_range:

        # This function plots f(x), the interpolating polynomial, and the associated error
        # points can be np.linspace to equidistant points or Chebyshev to get Chebyshev points

        # These two lines generate the data that will be used in the interpolation
        x = points(xmin,xmax,n)
        y = f(x)

        # These two lines compute the exact value of the function being interpolated on a finer grid
        xe = np.linspace(xmin,xmax,100)
        ye = f(xe)

        # This two lines build the interpolant chosen, where the name of the method is
        # passed as parameter usign the variable 'method'.
        time_construction = %timeit -o -q -n 10 -r 5 method(x,y)
        construction_times.append(time_construction.average)
        D = method(x,y)

        time_evaluation = %timeit -o -q -n 10 -r 5 EvaluateInterpolant(D, xe)
        evaluation_times.append(time_evaluation.average)
        yi = EvaluateInterpolant(D, xe)

        if max(abs(ye-yi))<1e-16:
            interpolation_errors.append(1e-16)
        else:
            interpolation_errors.append(max(abs(ye-yi)))

    return construction_times, evaluation_times, interpolation_errors

def analysis_error_vs_time_interpolation_algorihtms(f):
    # List of methods
    list_of_methods = [('Vandermonde',Vandermonde),('Lagrange',Lagrange),('Barycentric',Barycentric),('NewtonDD',NewtonDD),('SincInterpolator',SincInterpolator),('PiecewiseLinear',PiecewiseLinear)]
    # List of type of points
    list_of_points = [('equalspaced',np.linspace), ('Chebyshev',Chebyshev_points)]
    list_output_data = []
    k = 1
    for name_method, method in list_of_methods:
        for points_name, points in...


I am working on the file "SC1v2/Bonus - 05 - Newton's divided differences, Sinc and piecewiselinear interpolations.ipynb".
The first task is:
<div id='NewtonDD' />

Newton's Divided Difference
Back to TOC
In this interpolation method we will use divided differences to calculate the coefficients of our interpolation polynomial. Given a set of $n$ data points $(x_1,y_1),...,(x_n,y_n)$, the Newton polynomial is:
$$ p(x) = \sum^n_{i=1} (f[x_1 ... x_i] \cdot \prod^{i-1}{j=1} (x-x_j)) ,$$
where $ \prod^{0}{j=1} (x-x_j) = 0 $, and:
$$ f[x_i] = y_i $$
$$ f[x_j...x_i] = \frac{f[x_{j+1}...x_i]-f[x_j...x_{i-1}]}{x_i-x_j}$$
Can you write Python code for it?

def Divided_Differences(x, y):
    dd = np.array([y])
    n = len(x)
    for i in range(n-1):
        ddi = []
        for a in range(n-i-1):
            ddi.append((dd[i][a+1]-dd[i][a])/(x[a+i+1]-x[a]))
        ddi = np.append(ddi,np.full((n-len(ddi),),0.0))
        dd = np.append(dd,[ddi],axis=0)
    return np.array(dd)

def NewtonDD(x, y, show=False):
    # We calculate the divided differences and store them in a data structure
    dd = Divided_Differences(x,y)
    n = len(x)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: '); print([(x[i],y[i]) for i in range(n)])
        xS = sym.Symbol('x')
        N = np.dot(dd[:,0],np.append([1],[np.prod(xS-x[:i]) for i in range(1,n)]))
        print('Interpolation Function: ');
        print(N)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'Method':'Newton',
         'PrecomputedData': dd[:,0],
         'x': x,
         'y': y,
         'n': n}
    return D

x = np.linspace(0,1,7)
y = np.sin(7*x)
D = NewtonDD(x, y, show=True)
PlotInterpolant(D,ylim=None)


I am working on the file "SC1v2/Bonus - 05 - Newton's divided differences, Sinc and piecewiselinear interpolations.ipynb".
The first task is:
<div id='convergence' />

Convergence analysis
Back to TOC
The next set of experiments show a convergence analysis comparing equalspaced points to Chebychev points.
The $x$-axis is the number of points used and the $y$-axis is the maxmimum error obtain on the interval in logarithmic scale.
The experiment are the followings:
1. $f(x)=x^{10}$. In this case both sets of points perform similarly, however there is a significant drop in the error when $n$ is equal to $11$. This means, the error reach a minimum of $10^{-14}$ app at that number points. Why?
2. $f(x)=|x^3|$. This looks like a naive function but it is not. The equalspace points start to decrease from an error of $10^{0}$ until an error app $10^{-3}$ with approximately 10 points, it then start to increase the error. On the other hand, the Chebychev points show a decresing behavior upto $n=50$ but it only reaches an error of app $10^{-5}$, which is too high. What is the issue with this funcion?
3. $f(x)=\exp(-(x^{-2}))$. This is also a tricky function bevcause involves a division by $0$ when we evaluate it at $x=0$, but it is well defined at that points. Similarly to the first example, the equalspaced points reduce the error upto $n=10$, and then start to increase. For the Chebychev points, they decrease upto 40 points and the also show a slight increment in the error. What is the value of $f(x)$ at $x=0$?
4. $f(x)=\dfrac{1}{1+x^2}$. In this case both cases decrease the error but faster for the Chebyshev points. The main difference is that the minimum error reached for equalspaced points is about $10^{-8}$ and $10^{-13}$ for the Chebyshev points, in both cases about $n=37$.
5. $f(x)=\sin^3(x)$. In this case both cases reached an error $10^{-14}$ with $n=20$, then for equalspaced points start to increase the error right away but for Chebyshev points it starts to increase at $n=40$.
Answers to the previous questions:
1. The reason is that when $n=11$ we will be interpolation an polynomial of degree $10$ with a polynomial of degree $10$, so we can reproduce...
Can you write Python code for it?

my_functions=[lambda x: x**10,
              lambda x: np.abs(x**3),
              lambda x: np.exp(-(x**-2)),
              lambda x: 1/(1+x**2),
              lambda x: np.sin(x)**3]
labels = ["x^{10}",
          "|x^3|",
          "\exp(-x^{-2})",
          "1/(1+x^2)",
          "\sin^3(x)"]
data=zip(labels,my_functions)

radio_button_function=RadioButtons(
    options=list(data),
    description='Function:',
    disabled=False
)

radio_button_interpolation_algorithm=RadioButtons(
    options=[('Vandermonde',Vandermonde),('Lagrange',Lagrange),('Barycentric',Barycentric),('NewtonDD',NewtonDD),('SincInterpolator',SincInterpolator),('PiecewiseLinear',PiecewiseLinear)],
    value=Barycentric,
    description='Algorithm:',
    disabled=False
)

def convergence_study(my_function, method):
    n=100
    shift=2
    n_points=np.arange(shift,n)
    max_error=np.zeros(n-shift)
    max_error_es=np.zeros(n-shift)
    for i in n_points:
        if method!=SincInterpolator:
            max_error[i-shift] = Error(my_function, i, -1, 1, method, Chebyshev_points, plot_flag=False)
        else:
            max_error[i-shift] = 1
        max_error_es[i-shift] = Error(my_function, i, -1, 1, method, points=np.linspace, plot_flag=False)
    axis=plt.figure(figsize=(8,8))
    if method!=SincInterpolator:
        plt.semilogy(n_points,max_error,'bd',label='Chebyshev points',markersize=10)
    plt.semilogy(n_points,max_error_es,'r.',label='Equalspaced poins',markersize=16)
    plt.ylim(10**-16,10**4)
    plt.grid(True)
    plt.title('Interpolation Error')
    plt.xlabel('Number of points used in the interpolation')
    plt.ylabel('Max error on domain')
    plt.legend(loc='best')
    plt.show()
interact(convergence_study,my_function=radio_button_function, method=radio_button_interpolation_algorithm)


I am working on the file "SC1v2/Bonus - 05 - Newton's divided differences, Sinc and piecewiselinear interpolations.ipynb".
The first task is:
<div id='intro' />

Introduction
Back to TOC
In this Bonus jupyter notebook we add the Newton divided differences algorithm, Sinc interpolation and piecewise linear interpolation for comparison purposes.
Can you write Python code for it?

# Function that evaluates the xi's points in the polynomial
def EvaluateInterpolant(D, xx):
    if D['Method']=='Vandermonde':
        p = lambda x: np.dot(D['PrecomputedData'],np.power(x,np.arange(D['n'])))
    elif D['Method']=='Lagrange':
        p = lambda x: np.dot(D['PrecomputedData'],[np.prod(x - np.delete(D['x'],j)) for j in range(D['n'])])
    elif D['Method']=='Barycentric':
        wi = D['PrecomputedData']
        xi = D['x']
        yi = D['y']
        p = lambda x: np.sum(yi*wi/(x-xi))/np.sum(wi/(x-xi)) if len(xi[xi == x])==0 else np.ndarray.item(yi[xi == x])
    elif D['Method']=='Newton':
        p = lambda x: np.dot(D['PrecomputedData'],np.append([1],[np.prod(x-D['x'][:j]) for j in range(1,D['n'])]))
    elif D['Method']=='SincInterpolator':
        p = lambda x: np.dot([np.sinc((x-xi)/D['h']) for xi in D['x']],D['y'])
    elif D['Method']=='PiecewiseLinear':
        m = D['m']
        n = D['n']
        xi = D['x']
        yi = D['y']
        def p(x):
            for i in range(n-1):
                if xi[i]<= x and x<xi[i+1]:
                    return yi[i]+m[i]*(x-xi[i])
            # This is the last case, if x==x[-1]
            return yi[-1]
    return np.array([p(x) for x in xx])

# Function that shows the data points and the function that interpolates them.
def PlotInterpolant(D,ylim=None):
    xi = np.linspace(min(D['x']),max(D['x']),1000)
    yi = EvaluateInterpolant(D,xi)
    fig = plt.figure(figsize=(8,8))
    plt.plot(D['x'],D['y'],'ro',label='Interpolation points')
    plt.plot(xi,yi,'b-',label='$p(x)$')
    plt.xlim(min(xi)-0.5, max(xi)+0.5)
    if ylim:
        plt.ylim(ylim[0], ylim[1])
    else:
        plt.ylim(min(yi)-0.5, max(yi)+0.5)
    plt.grid(True)
    plt.legend(loc='best')
    plt.xlabel('$x$')
    plt.show()


I am working on the file "SC1v2/Bonus - 05 - Newton's divided differences, Sinc and piecewiselinear interpolations.ipynb".
The first task is:
Another set of previous definitions
Back to TOC
Can you write Python code for it?

def Chebyshev(xmin,xmax,n=5):
    # This function calculates the n Chebyshev points and plots or returns them depending on ax
    ns = np.arange(1,n+1)
    x = np.cos((2*ns-1)*np.pi/(2*n))
    y = np.sin((2*ns-1)*np.pi/(2*n))
    plt.figure(figsize=(10,5))
    plt.ylim(-0.1,1.1)
    plt.xlim(-1.1,1.1)
    plt.plot(np.cos(np.linspace(0,np.pi)),np.sin(np.linspace(0,np.pi)),'k-')
    plt.plot([-2,2],[0,0],'k-')
    plt.plot([0,0],[-1,2],'k-')
    for i in range(len(y)):
        plt.plot([x[i],x[i]],[0,y[i]],'r-')
        plt.plot([0,x[i]],[0,y[i]],'r-')
    plt.plot(x,[0]*len(x),'bo',label='Chebyshev points')
    plt.plot(x,y,'ro')
    plt.xlabel('$x$')
    plt.title('n = '+str(n))
    plt.grid(True)
    plt.legend(loc='best')
    plt.show()
def Chebyshev_points(xmin,xmax,n):
    ns = np.arange(1,n+1)
    x = np.cos((2*ns-1)*np.pi/(2*n))
    #y = np.sin((2*ns-1)*np.pi/(2*n))
    return (xmin+xmax)/2 + (xmax-xmin)*x/2

def Error(f, n, xmin, xmax, method=Lagrange, points=np.linspace, plot_flag=True):
    # This function plots f(x), the interpolating polynomial, and the associated error
    # points can be np.linspace to equidistant points or Chebyshev to get Chebyshev points

    # These two lines generate the data that will be used in the interpolation
    x = points(xmin,xmax,n)
    y = f(x)

    # These two lines compute the exact value of the function being interpolated on a finer grid
    xe = np.linspace(xmin,xmax,200)
    ye = f(xe)

    # This two lines build the interpolant chosen, where the name of the method is
    # passed as parameter usign the variable 'method'.
    D = method(x,y)
    yi = EvaluateInterpolant(D, xe)

    if plot_flag:
        plt.figure(figsize=(5,10))
        f, (ax1, ax2) = plt.subplots(1, 2, figsize=(15,5), sharey = False)

        ax1.plot(xe, ye,'k-', label='f(x)')
        ax1.plot(x, y,'ro', label='Interpolation points',markersize=10)
        ax1.plot(xe, yi,'b-', label='Interpolation')
        ax1.set_xlim(xmin-0.5,xmax+0.5)
       ...


I am working on the file "SC1v2/Bonus - 05 - Newton's divided differences, Sinc and piecewiselinear interpolations.ipynb".
The first task is:
Questions about Newton's DD:
- What is the main problem using this method (and Lagrange)? How can you fix it? A: A problem with polynomial interpolation with equispaced date is the Runge phenomenon and can be handle with Chebyshev points
- What to do when a new point is added? A: Pro, is not necessary re-calculate the whole polynomial only a small piece
<div id='SincInterpolator' />

Sinc interpolator
Back to TOC
The Sinc interpolator is not a polynomial interpolator but it is an interesting one to know.
It is useful for equalspaced data $x_i$ and if has the following form:
$$
g(x)=\sum_{i=1}^n y_i\,\text{sinc}\left(\frac{x-x_i}{h}\right),
$$
where $h=x_{i+1}-x_{i}$ and $\text{sinc}(x)=\dfrac{\sin(\pi\,x)}{\pi\,x}$.
Can you write Python code for it?

def SincInterpolator(x, y, show=False):
    # We first check the data in equalspaced
    dx = np.diff(x)
    if np.abs(np.max(dx)-np.min(dx))>1e-10:
        raise ValueError("SincInterpolator: Data not equalspaced")

    # Computing h = x[i+1]-x[i]
    h = dx[0]
    n = len(x)

    # The function shows the data if the flag is true
    if show:
        print('Data Points: ');
        print([(x[i],y[i]) for i in range(n)])
        xS = sym.Symbol('x')
        SI = np.dot([sym.sinc((xS-xi)/h) for xi in x],y)
        print('Interpolation Function: ');
        print(SI)

    # Finally, we return a data structure with our interpolating polynomial
    D = {'Method': 'SincInterpolator',
         'h': h,
         'x': x,
         'y': y}
    return D

D = SincInterpolator(x, y, show=True)
PlotInterpolant(D,ylim=None)


I am working on the file "SC1v2/Bonus - 07-08 - Gradient Descent and Nonlinear Least-Square.ipynb".
The first task is:
In the previous implementation we used the following notation:
- $n$: Number of iteration of Gradient Descent
- Black dot: Solution $[a_n,b_n]$ at $n$-th step of the Gradient Descent.
- Red-Yellow streamplot: Stream plot of the vector field generated by minus the Gradient of the error function $E(a,b)$
- Blue-Green contour: Contour plot of the error function $E(a,b)$.
Questions:
- Try: $\alpha=0.02$, $n=20$, and $m=10$. What do you observe? (keep initialization values of $a_0$ and $a_1$)
- Try: $\alpha=0.04$, $n=20$, and $m=10$. What do you observe? (keep initialization values of $a_0$ and $a_1$)
- Try: $\alpha=0.08$, $n=20$, and $m=10$. What do you observe? (keep initialization values of $a_0$ and $a_1$)
- Can we use a large value of $\alpha$?
- How is related $\alpha$ and the iteration number $n$?
<div id='GD_2D_NonLinearLeastSquare' />

Gradient Descent for a 2D nonlinear least-square problem
Back to TOC
In this case, we will explore the use of the the Gradient Descent algorithm applied to a nonlinear least-square problem with an exponential fit.
Let the function to be fit be,
$$
\begin{equation}
    y(t) = a\,\exp(b\,t),
\end{equation}
$$
where the error function is defined as follows,
$$
\begin{equation}
    E(a,b)=\sum_{i=1}^m (y_i-a\,\exp(b\,t_i))^2.
\end{equation}
$$
Now, to apply the Gradient Descent algorithm we need to compute the Gradient of $E(a,b)$ with respect to $a$ and $b$, which is the following,
$$
\begin{align}
    \frac{\partial E}{\partial a} &= \sum_{i=1}^m 2\,\exp(b\,t_i)(a\,\exp(b\,t_i)-y_i),\
    \frac{\partial E}{\partial b} &= \sum_{i=1}^m 2\,a\,\exp(b\,t_i)\,t_i\,(a\,\exp(b\,t_i)-y_i).
\end{align}
$$
As you may expect, this approach may create very large values for the gradient, which will be very challenging to handle them numerically.
So, an alternative approach is the following,  which we will call it "The Variant":
- Select an initial guess, say $\mathbf{x}_0$
- Compute the direction of fastest decrease: $\mathbf{d}_0=-\nabla E(\mathbf{x}_0)$
- Update the approximation...
Can you write Python code for it?

def GD_2D_nonlinear(a0=0.75, b0=0.75, alpha=0, n=0, m=10, TheVariantFlag=False):

    # Building data.
    np.random.seed(0)
    a = 1.1
    b = 0.23
    y = lambda t: a*np.exp(b*t)
    T = 10
    ti = T*(np.random.rand(m)*2-1)
    yi = y(ti)+np.random.normal(loc=0, scale=0.1, size=m)

    # Defining the Gradient
    E = lambda a, b: np.sum(np.power(yi-a*np.exp(b*ti),2))
    G = lambda a, b: np.array([np.sum(2*np.exp(b*ti)*(a*np.exp(b*ti)-yi)), np.sum(2*a*np.exp(b*ti)*ti*(a*np.exp(b*ti)-yi))],dtype=float)
    # This fucntion will help us to evaluate the Gradient on the points (X[i,j],Y[i,j])
    def E_mG_XY(AA,BB):
        Z = np.zeros_like(AA)
        U = np.zeros_like(AA)
        V = np.zeros_like(AA)
        for i in range(m):
            for j in range(m):
                Z[i,j]=E(AA[i,j],BB[i,j])
                uv = -G(AA[i,j],BB[i,j])
                U[i,j] = uv[0]
                V[i,j] = uv[1]
        return Z, U, V

    # Plotting the function and its gradient.
    # Credits:
    #  https://matplotlib.org/stable/gallery/images_contours_and_fields/plot_streamplot.html
    #  https://scipython.com/blog/visualizing-a-vector-field-with-matplotlib/
    x = np.linspace(-3,3,m)
    AA, BB = np.meshgrid(x,x)
    fig = plt.figure(figsize=(14,10))
    Z, U, V = E_mG_XY(AA,BB)
    cont = plt.contour(AA,BB,Z, 10)
    stream = plt.streamplot(AA, BB, U, V, color=Z, linewidth=2, cmap='autumn', arrowstyle='->', arrowsize=2)
    fig.colorbar(stream.lines)
    fig.colorbar(cont)
    plt.scatter(a0, b0, s=300, marker='.', c='k')
    my_grad = G(a0,b0)
    my_title = r'$\alpha=$ %.4f, $E(a,b)=$ %.4f, $\nabla E(a,b)=$ [%.4f, %.4f]' %  (alpha, E(a0,b0), my_grad[0], my_grad[1])
    plt.title(my_title)

    # Computing steps of Gradient Descent
    if n>0:
        ab_output=np.zeros((n+1,2))
        z0 = np.array([a0,b0],dtype=float)
        z0[0] = a0
        z0[1] = b0
        ab_output[0,:]=z0
        # The Gradient Descent Algorithm
        for k in range(n):
            G_E_0=G(z0[0],z0[1])
          ...


I am working on the file "SC1v2/Bonus - 07-08 - Gradient Descent and Nonlinear Least-Square.ipynb".
The first task is:
Now, we will create an interactive use of the Gradient Descent in 1D where you could define the initial guess $x_0$, the scaling factor $\alpha$ and the iteration number.
In this numerical experiment we will the importance of the coefficient $\alpha$, and how it is related to the 'gradient' and the initial guess.
Can you write Python code for it?

def GD_1D(x0=2, alpha=1, n=0):

    # Defining the function using a 'lambda' definition and its derivative.
    f = lambda x: (x-2)*np.sin(2*x)+np.power(x,2)
    fp = lambda x: 2*x+2*(x-2)*np.cos(2*x)+np.sin(2*x)

    # Plotting the function and its derivative.
    xx = np.linspace(-3,3,1000)
    plt.figure(figsize=(14,7))

    ax = plt.subplot(1,2,1)
    plt.plot(xx,f(xx),'b-',label=r'$f(x)$')
    # Warning: The 'alpha' parameter for the plt.plot function corresponds to
    # a transparency parameter, it is not related to the alpha parameter of
    # the Gradient Descent explained before.
    plt.plot(xx,fp(xx),'r-',label=r"$f'(x)$", alpha=0.5)
    plt.grid(True)
    plt.xlabel('$x$')
    plt.title('Plot in linear scale')

    # Plotting outcome with no iterations
    plt.plot(x0,f(x0),'k.',markersize=10,label=r'$x_i$')
    plt.plot(x0,fp(x0),'m.',markersize=10,label=r"$f'(x_i)$: 'Gradient'")
    ax = plt.subplot(1,2,2)
    plt.semilogy(xx,np.abs(f(xx)),'b-',label=r"$|f(x)|$")
    plt.semilogy(xx,np.abs(fp(xx)),'r-',label=r"$|f'(x)|$", alpha=0.5)
    plt.grid(True)
    plt.xlabel('$x$')
    plt.title('Plot in logarithmic scale')
    plt.semilogy(x0,np.abs(f(x0)),'k.',markersize=10,label=r'$x_i$')
    plt.semilogy(x0,np.abs(fp(x0)),'m.',markersize=10,label=r"$|f'(x_i)|$: 'Gradient'")

    # Computing steps of Gradient Descent
    if n>0:
        xi_output=np.zeros(n+1)
        xi_output[0]=x0
        for k in range(n):
            fp_x0=fp(x0)
            x1 = x0-alpha*fp_x0
            xi_output[k+1]=x1
            x0 = x1
        ax = plt.subplot(1,2,1)
        plt.plot(xi_output,f(xi_output),'k.-',markersize=10,label=r'$x_i$')
        plt.plot(xi_output,fp(xi_output),'m.',markersize=10)
        ax = plt.subplot(1,2,2)
        plt.semilogy(xi_output,np.abs(f(xi_output)),'k.-',markersize=10,label=r'$x_i$')
        plt.semilogy(xi_output,np.abs(fp(xi_output)),'m.',markersize=10)

    # Plotting outcome
    ax = plt.subplot(1,2,1)
    plt.legend(loc='best')
    ax =...


I am working on the file "SC1v2/Bonus - 07-08 Weighted Least Squares.ipynb".
The first task is:

Introduction
Back to TOC
This jupyter notebook presents the notion of weigthed linear least square problems.
It first present the theoretical background, then a few small examples and finally makes a connection with the "Initial Example" from the section "Overdetermined Linear Systems of Equations" in the notebook "07_08_Least_Squares".
We strongly suggest the reader to review that example first.

Weighted least-square
Back to TOC
In this example we will consider we give a different weight to each term in the least square problem.
Mathematical, this can be seen as multiplying the least-square problem by a diagonal matrix, say $W=\text{diag}(w_1,w_2,\dots,w_m)$, with $w_i>0$.
For instance, consider the following least square example:
\begin{equation}
    \underbrace{\begin{bmatrix}
        1 & x_1 \
        1 & x_2 \
        1 & x_3 \
        \vdots & \vdots \
        1 & x_m
    \end{bmatrix}}{\displaystyle{A}}
    \underbrace{\begin{bmatrix}
        a\
        b
    \end{bmatrix}}{\mathbf{x}}
    =
    \underbrace{\begin{bmatrix}
        y_1 \
        y_2 \
        y_3 \
        \vdots\
        y_m
    \end{bmatrix}}{\displaystyle{\mathbf{b}}}.
\end{equation}
In this case, all the equations have the same weight.
Now, giving a different weight to each equation, we could do the following:
\begin{equation}
    \underbrace{W\,A}{\displaystyle{B}}\,\mathbf{x}=\underbrace{W\,\mathbf{b}}{\displaystyle{\mathbf{c}}},
\end{equation}
where, as mentioned before, we could define $W$ as follows,
\begin{equation}
    W=\text{diag}(w_1,w_2,\dots,w_m).
\end{equation}
So, when computing the quadratic error we obtain,
\begin{equation}
E=\left\|W\,\mathbf{b}- W\,A\,\mathbf{x}\right\|_2^2=\sum{i=1}^m w_i^2\,(y_i-a-b\,x_i)^2.
\end{equation}
This indicates that when we use a higher value of $w_i$ for some $i$'s, these equations will have a higher impact in the minimization.
From the normal equations point of view, this slightly modified problem will satisfy the following normal...
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 07-08 Weighted Least Squares.ipynb".
The first task is:
We clearly observe that the least square solutions are different!
This is consistent with the previous explanation.
<div id='extensionInitialExample' />

Extension of "Initial Example" in jupyter notebook "07_08_Least_Squares"
Back to TOC
In this example we will approximate $m$ points considering a linear relationship.
This means that we will have the data points $(x_i,y_i)$ for $i\in{1,2,\dots,m}$ and consider the relationshop $y=a_0+a_1\,b$.
The error that will be added follows a normal distribution, but we will consider we use the following weight matrix,
\begin{equation}
    W=\text{diag}(w,w,1,\dots,1).
\end{equation}
This means we will more weight to the first twio equations, this is arbitrarily, and it is only use to show what effects brings to the least square problem.
The example will only show the approximation output since we already saw all the other components in the original example.
Question to think before to modify the weight $w$: What would you expect to happen with the approximation if you use a large value for $w$?
Can you write Python code for it?

def showWeightedOutput(w=1):
    # Number of points to be used
    m = 10
    # Relationship considered
    fv = np.vectorize(lambda x, a0, a1: a0+a1*x)
    # Coefficients considered
    a0, a1 = 1, 4

    np.random.seed(0)
    # Standard deviation for the error
    sigma = 5e-1
    # Error to be added
    e = np.random.normal(0,sigma,m)

    # Generating data points
    x = np.linspace(0,1,m)
    y = fv(x,a0,a1)+e

    # Build the data matrix
    A = np.ones((m,2))
    A[:,1] = x
    # Setting up the right hand side
    b = np.copy(y)
    A[:2,:]*=w
    b[:2]*=w

    # Building and solving the normal equations
    # A^T A x_bar = A^T b
    x_bar = np.linalg.solve(A.T @ A, A.T @ b)
    # Showing the comparison between the "original function" and the "least-squared reconstructed approximation".
    # We added in red a "sample" of possible functions.
    # Notice that the colors used follow the description included in the classnotes.
    # This means to consider the following analogy:
    # blue: data points, this correspond to the right-hand-side vector "b".
    # red: this correspond to the sub-space generated by Ax, i.e. the span of the columns of A.
    # violet: This correspond to the least-square solution found.
    plt.figure(figsize=(10,10))
    for i in range(100):
        plt.plot(x,fv(x,x_bar[0]+np.random.normal(0,1),x_bar[1]+np.random.normal(0,1)),'r-',linewidth=1,alpha=0.2)
    plt.plot(x,fv(x,a0,a1),'k-',linewidth=8,alpha=0.8)
    plt.plot(x,fv(x,x_bar[0],x_bar[1]),'--',color='darkviolet',linewidth=4)
    plt.plot(x,fv(x,x_bar[0],x_bar[1]),'r.',markersize=20)
    plt.plot(x,y,'b.',markersize=10)
    plt.grid(True)
    plt.xlabel(r'$x$')
    plt.ylabel(r'$y$')
    plt.show()
interact_manual(showWeightedOutput,w=(0.01,101,0.01))


I am working on the file "SC1v2/Bonus - 09 - Sylvester Equation with GMRes.ipynb".
The first task is:
<div id='chal' />

Challenge: What do we need to change in our implementation of GMRes to be able to use the lambda function "Ax"?
Back to TOC
Can you write Python code for it?

# This is a very instructive implementation of GMRes.
def GMRes_Ax(A, b, x0=np.array([0.0]), m=10, flag_display=True, threshold=1e-12):
    n = len(b)
    if len(x0)==1:
        x0=np.zeros(n)
    r0 = b - np.dot(A, x0)
    nr0=np.linalg.norm(r0)
    out_res=np.array(nr0)
    Q = np.zeros((n,n))
    H = np.zeros((n,n))
    Q[:,0] = r0 / nr0
    flag_break=False
    for k in np.arange(np.min((m,n))):
        y = np.dot(A, Q[:,k])
        if flag_display:
            print('||y||=',np.linalg.norm(y))
        for j in np.arange(k+1):
            H[j][k] = np.dot(Q[:,j], y)
            if flag_display:
                print('H[',j,'][',k,']=',H[j][k])
            y = y - np.dot(H[j][k],Q[:,j])
            if flag_display:
                print('||y||=',np.linalg.norm(y))
        # All but the last equation are treated equally. Why?
        if k+1<n:
            H[k+1][k] = np.linalg.norm(y)
            if flag_display:
                print('H[',k+1,'][',k,']=',H[k+1][k])
            if (np.abs(H[k+1][k]) > 1e-16):
                Q[:,k+1] = y/H[k+1][k]
            else:
                print('flag_break has been activated')
                flag_break=True
            # Do you remember e_1? The canonical vector.
            e1 = np.zeros((k+1)+1)
            e1[0]=1
            H_tilde=H[0:(k+1)+1,0:k+1]
        else:
            H_tilde=H[0:k+1,0:k+1]
        # Solving the 'SMALL' least square problem.
        # This could be improved with Givens rotations!
        ck = np.linalg.lstsq(H_tilde, nr0*e1)[0]
        if k+1<n:
            x = x0 + np.dot(Q[:,0:(k+1)], ck)
        else:
            x = x0 + np.dot(Q, ck)
        # Why is 'norm_small' equal to 'norm_full'?
        norm_small=np.linalg.norm(np.dot(H_tilde,ck)-nr0*e1)
        out_res = np.append(out_res,norm_small)
        if flag_display:
            norm_full=np.linalg.norm(b-np.dot(A,x))
            print('..........||b-A\,x_k||=',norm_full)
            print('..........||H_k\,c_k-nr0*e1||',norm_small);
        if flag_break:
            if...


I am working on the file "SC1v2/Bonus - 09 - Sylvester Equation with GMRes.ipynb".
The first task is:
<div id='sylvester' />

Sylvester Equation
The Sylvester Equation has the following form matricial form:
$$A\,X+X\,B=C$$
where $A\in\mathbb{R}^{n\times n}$, $B\in\mathbb{R}^{n\times n}$, $C\in\mathbb{R}^{n\times n}$ and $X\in\mathbb{R}^{n\times n}$.
$A$, $B$ and $C$ are given, the problem is to find the matrix $X$.
See https://en.wikipedia.org/wiki/Sylvester_equation.
Thus, an algorithm as PALU o classical version of ietrative solvers can't be applied directly.
If you want to do so, please take a look to the vec operator: https://en.wikipedia.org/wiki/Vectorization_%28mathematics%29 , together with NumPy flatten(order='F') procedure and the NumPy Kronecker product of two arrays, np.kron.
Notice that if you really want to translate the Sylvester Equation into the tradicional form $\widehat{A}\,\widehat{\mathbf{x}}=\widehat{\mathbf{b}}$, just be aware that $\widehat{A}\in\mathbb{R}^{n^2\times n^2}$, $\widehat{\mathbf{x}}\in\mathbb{R}^{n^2}$, and $\widehat{\mathbf{b}}\in\mathbb{R}^{n^2}$.
This means that if $n$ is large, $n^2$ is huge!
And you may not have enough memory to store $\widehat{A}$.
Vectorization: A quick review
In Mathematics the vec operator is an operator that translate a matrix into a vector as follows:
$$
\text{vec}
\left(
\begin{bmatrix}
  a & b \
  c & d
\end{bmatrix}
\right)
=
\begin{bmatrix}
  a \
  c \
  b \
  d
\end{bmatrix}.
$$
This can be achieved with the flatten function from NumPy with the parameter F. Consider the following numerical example:
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 09 - Sylvester Equation with GMRes.ipynb".
The first task is:
So, since the residual is decreasing we expect convergence. Let's look at the following example,
Can you write Python code for it?

# Second TEST
n = 10
np.random.seed(0)
A = np.random.rand(n,n)+2*np.eye(n)
#print(np.linalg.eigvals(A))
B = np.random.rand(n,n)
#print(np.linalg.eigvals(B))
C = np.random.rand(n,n)

X_JGS=solve_JGS_iterative_Sylvester(A,B,C,10)


I am working on the file "SC1v2/Bonus - 09 - Sylvester Equation with GMRes.ipynb".
The first task is:
Unfortunately in this case it is divergent.
One alternativs is to implement Algorithm 2 and hope it will converge.
A better alternative is to use GMRes!!
<div id='GMResTest' />

Using the beautiful GMRes
Back to TOC
Can you write Python code for it?

# This function computes the 'matrix-vector' product of the matrix we don't have explicitly stored!!
def compute_matrix_vector_product(x,A,B,n):
    X = np.reshape(x,(n,n),order='F')
    out = np.dot(A,X)+np.dot(X,B)
    return out.flatten('F')
# This is part of the interface that SciPy requires.
Ax = lambda x: compute_matrix_vector_product(x,A,B,n)
# This is the famous 'afun'!!
afun = spla.LinearOperator((n**2, n**2), matvec=Ax)
# Just running GMRes
x, exitCode = spla.gmres(afun, C.flatten('F'), tol=1e-10)
# Just reshaping the
X_GMRes = np.reshape(x,(n,n),order='F')
print('residual: ',np.linalg.norm(np.dot(A,X_GMRes)+np.dot(X_GMRes,B)-C))
#print(X_GMRes)


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
<div id='example1s' />

Solution Example 1
Back to toc
Considere el siguiente BVP:
\begin{align}
    a\,y''(x)+b(x)\,y'(x)+c(x)\,y(x) & = f(x), \quad \text{para $x\in]0,1[$}\
    a\,(y(0) - y_0)& = 0,\
    y(1)    & = y_1,
\end{align}
donde $a\in\mathbb{R}$, $b(x)>0$, y $x\in[0,1]$
Caso $a\neq0$
FD:
Considere el siguiente BVP:
\begin{align}
    a\,y''(x)+b(x)\,y'(x)+c(x)\,y(x) & = f(x), \quad \text{para $x\in]0,1[$}\
    a\,(y(0) - y_0)& = 0,\
    y(1)    & = y_1,
\end{align}
donde $a\in\mathbb{R}$, $b(x)>0$, y $x\in[0,1]$
Answer: Consider that $x_i = \frac{i}{N-1}$ for $i\,{0,1,\dots,N-1}$, and $y(x_i) \approx w_i$, where we know that $w_{0}=y_0$ and $w_{N-1}=y_1$.
For simplicity we will consider $\mathbf{w}=[w_1,w_2,\dots,w_{N-2}]$.
The finite diference discretizations that we will use are the followings:
\begin{align}
    y''(x_i) &\approx \dfrac{w_{i+1}-2\,w_i+w_{i-1}}{h^2},\
    y'(x_i)  &\approx \dfrac{w_{i+1}-w_{i-1}}{2\,h}.
\end{align}
Thus, the discrete version of the ode at $x_i$ will be the following:
\begin{align}
    a\,y''(x_i)      &\approx a\,\dfrac{w_{i+1}-2\,w_i+w_{i-1}}{h^2},\
    b(x_i)\,y'(x_i)  &\approx b(x_i)\,\dfrac{w_{i+1}-w_{i-1}}{2\,h},\
    c(x_i)\,y(x_i)   &\approx c(x_i)\,w_i,\
    f(x_i)           &\approx f(x_i).
\end{align}
By using the unknowns vector $\mathbf{w}$ and the know vector $\mathbf{x}=[x_1,x_2,\dots,x_{N-2}]$ we can define the following matrices:
\begin{align}
    D_2 &=
    \begin{bmatrix}
        -2 &  1 & 0 & 0 & 0 & 0 & 0 \
         1 & -2 & 1 & 0 & 0 & 0 & 0 \
         0 &  1 & -2 & 1 & 0 & 0 & 0 \
         \vdots & \ddots & \ddots & \ddots & \ddots & \ddots & \ddots \
         0 &  0 & 0 & 0 & 1 & -2 & 1 \
         0 &  0 & 0 & 0 & 0 & 1 & -2 \
    \end{bmatrix},\
    D  &=
    \begin{bmatrix}
         0 & 1 & 0 & 0 & 0 & 0 & 0 \
         -1 & 0 & 1 & 0 & 0 & 0 & 0 \
         0 &  -1 & 0 & 1 & 0 & 0 & 0 \
         \vdots & \ddots & \ddots & \ddots & \ddots & \ddots & \ddots \
         0 &  0 & 0 & 0 & -1 & 0 & 1 \
         0 &  0 & 0 & 0 & 0 & -1 & 0 \
   ...
Can you write Python code for it?

# This function builds the h-less differentiation matrices for
# the approximation of the first and second derivatives.
# h-less means that it still needs to add the corresponding
# h coefficient in the approximation.
def build_D_D2(M):
    # First derivative - Central difference differentiation matrix
    D  = toeplitz(np.append(np.array([0, -1.]), np.zeros(M-2)),
                     np.append(np.array([0, 1.]), np.zeros(M-2)))
    # Second derivative - differentiation matrix
    D2 = toeplitz(np.append(np.array([-2, 1.]), np.zeros(M-2)))
    return D, D2
D , D2 = build_D_D2(5)
print('D: \n', D)
print('D2: \n', D2)


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
<div id='example2' />

Example 2
Back to toc
Considere el siguiente BVP:
\begin{align}
    y''(x)+3\,\exp(y(x)) & = 0, \quad \text{para $x \in]0,1[$}\
    y(0) &= 0,\
    y(1) &= 0,\
\end{align}
donde $x\in[0,1]$. Note que $y(x)=0$ no es una solución.
1. Construya un algoritmo basado en diferencias finitas que obtenga la aproximación numérica $y(x)$ considerando como initial guess $y_0(x)$. Utilice el método de Newton con GMRes para resolver el sistema de ecuaciones no-lineales asociado o el módulo newton_krylov de scipy, en este último caso no es necesario implementar build_jacobian_matrix_FD.
2. Construya un algoritmo basado en el método del disparo que obtenga la aproximación numérica $y(x)$ considerando como initial guess $y_0(x)$.
3. Resuelva con el algoritmo basado en diferencias finitas el BVP con $y_0(x)=0$ con N=20.
4. Resuelva con el algoritmo basado en el método del disparo el BVP con $y_0(x)=0$ con N=20.
5. ¿Obtiene aproximadamente la misma aproximación numérica en la pregunta 3 y 4?
6. Considere la siguiente familia de initial guesses $y_0^{[m]}(x)=m\,(x-x^2)$, para $m\in{-10,-9,\dots,10}$ y N=20.
    1. Resuelva con el algoritmo basado en diferencias finitas el BVP con cada $y_0^{[m]}(x)$.
    2. Resuelva con el algoritmo basado en el método del disparo el BVP con cada $y_0^{[m]}(x)$.
    3. ¿Se obtinen las mismas soluciones en cada caso?
    4. ¿Cuantas soluciones distintas se obtienen?
Hint: Take a look to the additional parameters requiered in the definitions of the functions below.
Can you write Python code for it?

'''
input:
yi        : (ndarray) vector donde se evalua la matrix Jacobiana
v         : (ndarray) vector que será multiplicado por la matriz Jacobiana

output:
Jv        : (ndarray) matrix Jacobiana evaluada en yi y multiplicada por v, es decir np.dot(J,v).
                    Esto es muy útil para acoplarlo con GMRes.
'''
def build_jacobian_matrix_FD(yj):
    # Your own code.
    return Jv

'''
input:
y0        : (callable) initial guess y0(x)
N         : (integer) número de puntos en la discretización espacial de x

output:
xi        : (ndarray) discretización equiespaciada de N puntos x
yi        : (ndarray) aproximación numérica de y(x) en los puntos xi
'''
def solve_nonlinear_ode_FD(y0, N):
    # Your own code.
    return xi, yi

'''
input:
y0        : (callable) initial guess y0(x)
N         : (integer) número de puntos en la discretización espacial de x

output:
xi        : (ndarray) discretización equiespaciada de N puntos x
yi        : (ndarray) aproximación numérica de y(x) en los puntos xi
'''
def solve_nonlinear_ode_SM(y0, N):
    # Your own code.
    return xi, yi


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
<div id='example3s' />

Example 3
Back to toc
Considere que tiene la siguiente colección de datos:
\begin{align}
    {(t_1, y_1),(t_2,y_2),\dots,(t_n,y_n)},
\end{align}
donde sabemos que $0 \leq t_i \leq 100$ para $i\in{1,2,\dots,n}$.
Una aproximación tradicional de mínimos cuadrados requeriría proponer una función $y(t)$, por ejemplo lineal $a+b\,t$, para minimizar el error cuadrático, $E=\sum_{i=1}^n \left(y(t_i)-y_i\right)^2$.
Esto nos entregaría los coeficientes de la estructura propuesta para $y(t)$.
La mayor desventaja de este procedimiento es que tenemos que conocer a priori la estructura algebraica de $y(t)$.
Una alternativa sería construir numéricamente una aproximación de $y(t)$, por ejemplo reemplazando la minimización por un problema de valor inicial, para lo cual se propone el siguiente IVP,
\begin{align}
    \dot{y}(t) &= \gamma\,\tanh\left(\dfrac{\displaystyle\sum_{t_i\in[t-T,t]} (y_i-y(t))}{\gamma}\right),\
    y(0)       &= y_1,
\end{align}
donde $\gamma=1$.
1. Implemente el solver.
2. ¿Se suaviza la data?
3. ¿Cómo depende la aproximación en función de $T$?
4. ¿Cómo depende la aproximación en función de $\gamma$?
Nota: Si el conjunto $t_i\in[t-T,t]$ es vacio, entonces el lado derecho de $\dot{y}(t)$ se considera que es $0$.
Can you write Python code for it?

'''
input:
N         : (integer) número de puntos en la discretización espacial del intervalo [0,100]
Tw        : (double)  ventana deslizante utilizada en la sumatoria [t-Tw,t]
gamma     : (double)  coeficiente usado en conjunto con tanh
Ti        : (double) tiempo inicial de simulación
Tf        : (double) tiempo final de simulación
ti        : (ndarray) ti data de entrada
yi        : (ndarray) yi data de entrada

output:
t_out        : (ndarray) discretización equiespaciada de N puntos t
y_out        : (ndarray) aproximación numérica de y(t) en los puntos ti
'''
def solve_almost_LS_IVP(N,Tw,gamma,Ti,Tf,ti,yi):
    # Your own code.
    return t_out, y_out

# Consider the following data
np.random.seed(0)
Ndata    = 1000
ti       = np.linspace(0,100,Ndata)
yi       = 0.2*np.cos(2*ti)+np.sin(0.1*ti)+0.1*np.random.rand(Ndata)

'''
input:
N         : (integer) número de puntos en la discretización espacial del intervalo [0,100]
Tw        : (double)  ventana deslizante utilizada en la sumatoria [t-Tw,t]
gamma     : (double)  coeficiente usado en conjunto con tanh
Ti        : (double) tiempo inicial de simulación
Tf        : (double) tiempo final de simulación
ti        : (ndarray) ti data de entrada
yi        : (ndarray) yi data de entrada

output:
t_out        : (ndarray) discretización equiespaciada de N puntos t
y_out        : (ndarray) aproximación numérica de y(t) en los puntos ti
'''
def solve_almost_LS_IVP(N,Tw,gamma,Ti,Tf,ti,yi):
    def my_f(t,y):
        ysample = yi[np.logical_and((t-Tw)<ti,ti<t)]
        if len(ysample)>=1:
            return gamma*np.tanh((np.sum(ysample)-len(ysample)*y)/gamma)
        else:
            return 0
    sol = solve_ivp(my_f,(Ti,Tf),(yi[0],),t_eval=np.linspace(Ti,Tf,N))
    t_out = sol.t
    y_out = sol.y[0]
    return t_out, y_out

def show_output_LS_IVP(Tw=3,gamma=1):
    N        = 10000
    Ti       = 0
    Tf       = 120
    t_out, y_out = solve_almost_LS_IVP(N,Tw,gamma,Ti,Tf,ti,yi)

    plt.figure(figsize=(16,8))
    plt.plot(ti,yi,'.',label=r'$y_i$')
   ...


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
Caso $a=0$
FD
Considere el siguiente BVP:
\begin{align}
    a\,y''(x)+b(x)\,y'(x)+c(x)\,y(x) & = f(x), \quad \text{para $x\in]0,1[$}\
    a\,(y(0) - y_0)& = 0,\
    y(1)    & = y_1,
\end{align}
donde $a\in\mathbb{R}$, $b(x)>0$, y $x\in[0,1]$
Answer:
In this case the BVP becomes:
\begin{align}
    b(x)\,y'(x)+c(x)\,y(x) & = f(x), \quad \text{para $x\in]0,1[$}\
    y(1)    & = y_1.
\end{align}
So, if we use forward difference we obtain:
\begin{equation}
    \dfrac{b(x_i)}{h} \left(w_{i+1}-w_{i}\right)
    +
    c(x_i)\,w_i
    =
    f(x_i).
\end{equation}
There is only one special case now, for $x_{N-2}$, it generates the following equations:
\begin{equation}
    \dfrac{b(x_{N-2})}{h} \left(w_{N-1}-w_{N-2}\right)
    +
    c(x_{N-2})\,w_{N-2}
    =
    f(x_{N-2}).
\end{equation}
But, since we know $w_{N-1}$, we obtain,
\begin{equation}
    \dfrac{b(x_{N-2})}{h} \left(-w_{N-2}\right)
    +
    c(x_{N-2})\,w_{N-2}
    =
    f(x_{N-2})-\dfrac{b(x_{N-2})}{h}\,w_{N-1}.
\end{equation}
This analysis allows us to write the discrete equation in the following way for the unknown vector $\mathbf{w}=[w_1,w_2,\dots,w_{N-2}]$ and $\mathbf{x}=[x_1,x_2,\dots,x_{N-2}]$:
\begin{equation}
    \dfrac{1}{h}\,\text{diag}(b(\mathbf{x}))\,D^{\text{F}}\,\mathbf{w}
    +
    \text{diag}(c(\mathbf{x}))\,\mathbf{w}
    =
    \begin{bmatrix}
        f(x_1)\
        f(x_2)\
        \vdots\
        f(x_{N-3})\
        f(x_{N-2})-\dfrac{b(x_{N-2})}{h}\,w_{N-1}.
    \end{bmatrix},
\end{equation}
but since $w_{N-1}=y_1$ we get,
\begin{equation}
    \dfrac{1}{h}\,\text{diag}(b(\mathbf{x}))\,D^{\text{F}}\,\mathbf{w}
    +
    \text{diag}(c(\mathbf{x}))\,\mathbf{w}
    =
    \begin{bmatrix}
        f(x_1)\
        f(x_2)\
        \vdots\
        f(x_{N-3})\
        f(x_{N-2})-\dfrac{b(x_{N-2})}{h}\,y_1.
    \end{bmatrix},
\end{equation}
where
\begin{equation}
    D^{\text{F}}
    =
    \begin{bmatrix}
         -1 & 1 & 0 & 0 & 0 & 0 & 0 \
         0 & -1 & 1 & 0 & 0 & 0 & 0 \
         0 &  0 & -1 & 1 & 0 & 0 & 0 \
         \vdots & \ddots &...
Can you write Python code for it?

def build_DF(M):
    # First derivative - Forward difference differentiation matrix
    DF  = toeplitz(np.append(np.array([-1]), np.zeros(M-1)), np.append(np.array([0,1]), np.zeros(M-2)))
    return DF

def build_A_hat_and_b_hat(b,c,f,h,x,y1,N):
    DF = build_DF(N-1)
    x_interior = x[:-1]
    A_hat_N=(1/(h))*np.dot(np.diag(b(x_interior)),DF)+np.diag(c(x_interior))
    b_hat_N = f(x_interior)
    b_hat_N[-1] = b_hat_N[-1]-(b(x_interior[-1])/(h))*y1
    return A_hat_N, b_hat_N

A_hat_N, b_hat_N = build_A_hat_and_b_hat(b,c,f,h,x,y1,N)
w = np.linalg.solve(A_hat_N, b_hat_N)
w = np.append(w,y1)

plt.figure(figsize=(16,8))
plt.plot(x,w,'.',label=r'$w_i$')
plt.xlabel(r'$x_i$')
plt.grid(True)
plt.legend(loc='best')
plt.ylim([0,1.1])
plt.show()


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
Notice that in this particular case the solution for the case when $a\neq 0$ and $a=0$ are close to each other, but they are not the same.
In particular, the value at $x=0$ is different.
<div id='example2s' />

Solution Example 2
Back to toc
Considere el siguiente BVP:
\begin{align}
    y''(x)+3\,\exp(y(x)) & = 0, \quad \text{para $x \in]0,1[$}\
    y(0) &= 0,\
    y(1) &= 0,\
\end{align}
donde $x\in[0,1]$. Note que $y(x)=0$ no es una solución.
1. Construya un algoritmo basado en diferencias finitas que obtenga la aproximación numérica $y(x)$ considerando como initial guess $y_0(x)$. Utilice el método de Newton con GMRes para resolver el sistema de ecuaciones no-lineales asociado o el módulo newton_krylov de scipy, en este último caso no es necesario implementar build_jacobian_matrix_FD.
2. Construya un algoritmo basado en el método del disparo que obtenga la aproximación numérica $y(x)$ considerando como initial guess $y_0(x)$.
3. Resuelva con el algoritmo basado en diferencias finitas el BVP con $y_0(x)=0$ con N=20.
4. Resuelva con el algoritmo basado en el método del disparo el BVP con $y_0(x)=0$ con N=20.
5. ¿Obtiene aproximadamente la misma aproximación numérica en la pregunta 3 y 4?
6. Considere la siguiente familia de initial guesses $y_0^{[m]}(x)=m\,(x-x^2)$, para $m\in{-10,-9,\dots,10}$ y N=20.
    1. Resuelva con el algoritmo basado en diferencias finitas el BVP con cada $y_0^{[m]}(x)$.
    2. Resuelva con el algoritmo basado en el método del disparo el BVP con cada $y_0^{[m]}(x)$.
    3. ¿Se obtinen las mismas soluciones en cada caso?
    4. ¿Cuantas soluciones distintas se obtienen?
FD
In this case we will re-use $D_2$ from the previous analysis, so the discrete equation becomes:
The discrete equation at $x_i$ is the following:
\begin{equation}
    \dfrac{1}{h^2} \left(w_{i+1}-2\,w_i+w_{i-1}\right)
    +3\,\,\exp(w_i)
    =
    0.
\end{equation}
There are two special cases, for $x_1$ and $x_{N-2}$, they generate the following equations:
\begin{align}
    \dfrac{1}{h^2}...
Can you write Python code for it?

N = 100
x = np.linspace(0,1,N)
h = 1/(N-1)
_, D2 = build_D_D2(N-2)
def F(w):
    return np.dot(D2,w)/(h**2)+3*np.exp(w)

w0 = lambda m: m*(x[1:-1]-np.power(x[1:-1],2))
# First solution
w = newton_krylov(F,w0(0))
w = np.append(0,w)
w = np.append(w,0)

# First solution, notice that the initial guess is different
w2 = newton_krylov(F,w0(8))
w2 = np.append(0,w2)
w2 = np.append(w2,0)

plt.figure(figsize=(16,8))
plt.plot(x,w,'.',label=r'$w$')
plt.plot(x,w2,'.',label=r'$w2$')
plt.legend(loc='best')
plt.xlabel(r'$x_i$')
plt.grid(True)
plt.show()


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
SM
Considere el siguiente BVP:
\begin{align}
    a\,y''(x)+b(x)\,y'(x)+c(x)\,y(x) & = f(x), \quad \text{para $x\in]0,1[$}\
    a\,(y(0) - y_0)& = 0,\
    y(1)    & = y_1,
\end{align}
donde $a\in\mathbb{R}$, $b(x)>0$, y $x\in[0,1]$
We first need to rewrite the BVP as a dynamical system and consider $x\rightarrow t$:
\begin{align}
    w_1(t) &= y(t),\
    w_2(t) &= y'(t),\
\end{align}
so,
\begin{align}
    \dot{w}_1 &= y'(t) = w_2,\
    \dot{w}_2(t) &= y''(t)\
                 &= \dfrac{1}{a}\left(f(t) - b(t)\,w_2-c(t)\,w_1\right),\
    w_1(0)   &= y_0,\
    w_2(0)   &= \alpha.
\end{align}
Can you write Python code for it?

# RHS of dynamical system
def my_f1(t,w,a,b,c,f):
    w1 = w[0]
    w2 = w[1]
    w1dot = w2
    w2dot = (f(t)-b(t)*w2-c(t)*w1)/a
    return np.array([w1dot,w2dot])

# Function to be used to apply the Shooting Method
def F_SM_1(alpha,a,b,c,f,y0,y1,N):
    t = np.linspace(0,1,N)
    initial_condition = np.zeros(2)
    initial_condition[0] = y0
    initial_condition[1] = alpha
    sol = solve_ivp(my_f1,(0,1),initial_condition,t_eval=t,args=(a,b,c,f))
    return sol.y[0,-1]-y1

F_root_1 = lambda alpha: F_SM_1(alpha,a,b,c,f,y0,y1,N)
alpha_r = root(F_root_1, 0.).x[0]

sol = solve_ivp(my_f1,(0,1),np.array([y0,alpha_r]),t_eval=np.linspace(0,1,N),args=(a,b,c,f))

plt.figure(figsize=(16,8))
plt.plot(sol.t,sol.y[0,:],'rd',label='SM',alpha=0.5)
plt.plot(x,w,'.',label=r'$w_i$')
plt.legend(loc='best')
plt.xlabel(r'$x_i$')
plt.grid(True)
plt.show()


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
The discrete equation at $x_i$ is the following:
\begin{equation}
    \dfrac{a}{h^2} \left(w_{i+1}-2\,w_i+w_{i-1}\right)
    +
    \dfrac{b(x_i)}{2\,h} \left(w_{i+1}-w_{i-1}\right)
    +
    c(x_i)\,w_i
    =
    f(x_i).
\end{equation}
There are two special cases, for $x_1$ and $x_{N-2}$, they generate the following equations:
\begin{align}
    \dfrac{a}{h^2} \left(w_{2}-2\,w_1+w_0\right)
    +
    \dfrac{b(x_1)}{2\,h} \left(w_{2}-w_{0}\right)
    +
    c(x_1)\,w_1
    &=
    f(x_1),\
    \dfrac{a}{h^2} \left(w_{N-1}-2\,w_{N-2}+w_{N-3}\right)
    +
    \dfrac{b(x_{N-2})}{2\,h} \left(w_{N-1}-w_{N-3}\right)
    +
    c(x_{N-2})\,w_{N-2}
    &=
    f(x_{N-2}).
\end{align}
But, since we know $w_0$ and $w_{N-1}$, they become,
\begin{align}
    \dfrac{a}{h^2} \left(w_{2}-2\,w_1\right)
    +
    \dfrac{b(x_1)}{2\,h} w_{2}
    +
    c(x_1)\,w_1
    &=
    f(x_1)-\dfrac{a}{h^2} w_0+\dfrac{b(x_1)}{2\,h}\,w_0,\
    \dfrac{a}{h^2} \left(-2\,w_{N-2}+w_{N-3}\right)
    +
    \dfrac{b(x_{N-2})}{2\,h} \left(-w_{N-3}\right)
    +
    c(x_{N-2})\,w_{N-2}
    &=
    f(x_{N-2})-\dfrac{a}{h^2}\,w_{N-1}-\dfrac{b(x_{N-2})}{2\,h}\,w_{N-1}.
\end{align}
This analysis allows us to write the discrete equation in the following way for the unknown vector $\mathbf{w}=[w_1,w_2,\dots,w_{N-2}]$ and $\mathbf{x}=[x_1,x_2,\dots,x_{N-2}]$:
\begin{equation}
    \dfrac{a}{h^2}\,D_2\,\mathbf{w}
    +
    \dfrac{1}{2\,h}\,\text{diag}(b(\mathbf{x}))\,D\,\mathbf{w}
    +
    \text{diag}(c(\mathbf{x}))\,\mathbf{w}
    =
    \begin{bmatrix}
        f(x_1)-\dfrac{a}{h^2} w_0+\dfrac{b(x_1)}{2\,h}\,w_0\
        f(x_2)\
        \vdots\
        f(x_{N-3})\
        f(x_{N-2})-\dfrac{a}{h^2}\,w_{N-1}-\dfrac{b(x_{N-2})}{2\,h}\,w_{N-1}
    \end{bmatrix},
\end{equation}
but since $w_0=y_0$ and $w_{N-1}=y_1$ we get,
\begin{align}
    \dfrac{a}{h^2}\,D_2\,\mathbf{w}
    +
    \dfrac{1}{2\,h}\,\text{diag}(b(\mathbf{x}))\,D\,\mathbf{w}
    +
    \text{diag}(c(\mathbf{x}))\,\mathbf{w}
    &=
    \begin{bmatrix}
        f(x_1)-\dfrac{a}{h^2}...
Can you write Python code for it?

# Data definition
N=100
a = 1
b = lambda x: 10+x
c = lambda x: -10+x
f = lambda x: 1+20*np.sin(10*x)
x = np.linspace(0,1,N)
h = 1/(N-1)
y0 = 0
y1 = 1

def build_A_and_b(a,b,c,f,h,x,y0,y1,N):
    D, D2 = build_D_D2(N-2)
    x_interior = x[1:-1]
    A_N=(a/(h**2))*D2+(1/(2*h))*np.dot(np.diag(b(x_interior)),D)+np.diag(c(x_interior))
    b_N = f(x_interior)
    b_N[0] = b_N[0]-(a/(h**2))*y0+(b(x_interior[0])/(2*h))*y0
    b_N[-1] = b_N[-1]-(a/(h**2))*y1-(b(x_interior[-1])/(2*h))*y1
    return A_N, b_N

A_N, b_N = build_A_and_b(a,b,c,f,h,x,y0,y1,N)
w = np.linalg.solve(A_N,b_N)
w = np.append(y0,w)
w = np.append(w,y1)

plt.figure(figsize=(16,8))
plt.plot(x,w,'.',label=r'$w_i$')
plt.xlabel(r'$x_i$')
plt.grid(True)
plt.legend(loc='best')
plt.show()


I am working on the file "SC1v2/Bonus - 11 - BVP linear and nonlinear with Finite Difference and the Shooting Method.ipynb".
The first task is:
Warning:
The following numerical solutions give the 'core' to produce the required answers for the questions presented, you should still work on how to put the components together to generate the particular answers requested.
<div id='example1' />

Example 1
Back to toc
Considere el siguiente BVP:
\begin{align}
    a\,y''(x)+b(x)\,y'(x)+c(x)\,y(x) & = f(x), \quad \text{para $x\in]0,1[$}\
    a\,(y(0) - y_0)& = 0,\
    y(1)    & = y_1,
\end{align}
donde $a\in\mathbb{R}$, $b(x)>0$, y $x\in[0,1]$
1. Construya un solver para el problema anterior considerando todos los posibles casos utilizando diferencias finitas.

Construya un solver para el problema anterior considerando todos los posibles casos utilizando el método del disparo.

Hint: Does you solvers really consider all the cases? Even when $a=0$?

Obtenga con cada solver: $\int_0^1 y(x)\,dx$, $\int_0^1 y'(x)\,dx$, y $\int_0^1 y''(x)\,dx$ usando los siguientes algoritmos:
Trapecio
Punto Medio
Simpson
Cuadratura Gaussiana
Algebraicamente

Hint2: Take a look to the additional parameters requiered in the definitions of the functions below.
Can you write Python code for it?

'''
input:
a         : (double) coeficiente 'a'
b         : (callable) función b(x)>0
c         : (callable) función c(x)
f         : (callable) función f(x)
y0        : (double) y_0
y1        : (double) y_1
N         : (integer) número de puntos en la discretización espacial de x

output:
xi        : (ndarray) discretización equiespaciada de N puntos x
yi        : (ndarray) aproximación numérica de y(x) en los puntos xi
'''
def find_y_FD(a, b, c, f, y0, y1, N):
    # Your own code.
    return xi, yi

'''
input:
a         : (double) coeficiente 'a'
b         : (callable) función b(x)>0
c         : (callable) función c(x)
f         : (callable) función f(x)
y0        : (double) y_0
y1        : (double) y_1
N         : (integer) número de puntos en la discretización espacial de x

output:
xi        : (ndarray) discretización equiespaciada de N puntos x
yi        : (ndarray) aproximación numérica de y(x) en los puntos xi
'''
def find_y_SM(a, b, c, f, y0, y1, N):
    # Your own code.
    return xi, yi


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
<div id='doublependulum' />

Double Pendulum and chaos
Back to TOC
Source: https://demonstrations.wolfram.com/DoublePendulum/ and https://scienceworld.wolfram.com/physics/DoublePendulum.html
In this case the components are pretty much the same, the main difference is that now we have a second pendulum that it attached to the mass $m_1$, Figure below for an sketch of how the second pendulum is included.
Can you write Python code for it?

plt.figure(figsize=(8,8))
ax=plt.gca()
theta0=np.pi/4
theta1=np.pi/3
x1=np.sin(theta0)
y1=-np.cos(theta0)
x2=x1+0.7*np.sin(theta1)
y2=y1-0.7*np.cos(theta1)
plt.plot([0, x1],[0, y1],'-k')
plt.plot([x1, x2],[y1, y2],'-k')
plt.scatter(x1, y1, s=200, marker='o', c='b')
plt.scatter(x2, y2, s=200, marker='o', c='r')
plt.scatter(0, 0, s=200, marker='o', c='k')
plt.xlim([-1.5,2.5])
plt.ylim([-1.5,0.5])
plt.grid(True)
patches=[]
wedge1 = mpatches.Wedge((0, 0), 0.7, 270, 270+45, ec='none')
wedge2 = mpatches.Wedge((x, y), 0.7, 270, 270+60, ec='none')
patches.append(wedge1)
patches.append(wedge2)
collection = PatchCollection(patches, cmap=plt.cm.hsv, alpha=0.3)
ax.add_collection(collection)
plt.text(0.1, -0.4, r'$\theta_1$', fontsize=20)
plt.text(x1+0.1, y1-0.3, r'$\theta_2$', fontsize=20)
plt.text(0.8, -0.7, r'$m_1$', fontsize=20)
plt.text(x2+0.1, y2, r'$m_2$', fontsize=20)
plt.text(0.35, -0.25, r'$l_1$', fontsize=20)
plt.text(0.5*(x1+x2)+0.1, 0.5*(y1+y2), r'$l_2$', fontsize=20)
plt.show()


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
Here we add the perturbation $\delta$.
Please play with the values and analyze the output!
Can you write Python code for it?

delta = -0.1

y01=np.array([np.pi/2,np.pi/2,2,-2])
y02=np.array([np.pi/2,np.pi/2,2,-2+delta])

N = 1000
T = 20
args = (1,1,1,1,9.8)
t1, sol1 = solveDoblePendulum(T=T, N=N, y0=y01, args=args)
t2, sol2 = solveDoblePendulum(T=T, N=N, y0=y02, args=args)

def plotSeveralPendulums(k=10):

    plt.figure(figsize=(15,10))
    ax1 = plt.subplot(231)
    ax2 = plt.subplot(232)
    ax3 = plt.subplot(233)
    ax4 = plt.subplot(234)
    ax5 = plt.subplot(235)
    ax6 = plt.subplot(236)

    plotFastDoblePendulum(ax1, ax4, t1, sol1, T, k, args=args)
    plotFastDoblePendulum(ax2, ax5, t1, sol1, T, k, args=args)
    plotFastDoblePendulum(ax2, ax5, t2, sol2, T, k, args=args, color_1st_mass='m', color_2nd_mass='g')
    plotFastDoblePendulum(ax3, ax6, t2, sol2, T, k, args=args, color_1st_mass='m', color_2nd_mass='g')

    min_y = min(np.min(sol1[:]),np.min(sol2[:]))
    max_y = max(np.max(sol1[:]),np.max(sol2[:]))

    ax4.set_ylim([min_y, max_y])
    ax5.set_ylim([min_y, max_y])
    ax6.set_ylim([min_y, max_y])

    plt.show()


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
Notice that the original double pendulum is on the first column of plots, the perturbed double pendulum is on the third column, and we include the data for both of the in the second (central) column of plots.
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
Now solving the differential equation
Can you write Python code for it?

def solve_pendulum(N=1000, T=10,y0=np.array([3*np.pi/2, 0.0]),args=(9.8,1,1)):
    t = np.linspace(0, T, N)
    sol = odeint(myFirstPendulum, y0, t, args=args)
    return t, sol

'''def solve_pendulum0(N=1000):
    y0 = [3*np.pi/2, 0.0]
    t = np.linspace(0, 10, N)
    sol = odeint(myFirstPendulum, y0, t, args=(9.8,0.2,0.1))
    thetas = sol[:,0]
    return t, sol'''

t, sol = solve_pendulum()
thetas=sol[:,0]
omegas=sol[:,1]

plt.figure(figsize=(5,5))
plt.plot(t, thetas, 'b', label=r'$\theta$(t)')
plt.plot(t, omegas, 'g', label=r'$\omega$(t)')
plt.legend(loc='lower right',fontsize=20)
plt.xlabel('t')
plt.grid()
plt.title('What are we seeing?', fontsize=20)
plt.show()


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
Running the double pendulum numerical simulation
Here, we added the trajectory of both pendulums.
In the previous case, i.e. only one pendulum, the trajectory was always within a circle, however the second pendulum does not have to follow that!
Can you write Python code for it?


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
Showing Chaos in a double pendulum!
The main idea here is to run two simulations where the only difference between the numerical simulation is that only one initial conditions was perturbed respet to the otehr one.
This perturnation will be denoted as $\delta$ in the code.
See the next cell with comments.
Can you write Python code for it?

def solveDoblePendulum(T=10 , N=100, y0=np.array([np.pi/2,np.pi/2,2,-2]), args=(1,1,1,1,9.8)):

    l1, l2, m1, m2, g = args

    t  = np.linspace(0,T,N)

    #y0 = np.array([np.pi/2,np.pi/2,2,-2])
    sol = odeint(doublePendulum, y0, t, args=(l1, l2, m1, m2, g))
    return t, sol

def plotFastDoblePendulum(axT, axF, t, sol, T, k=0, args=(1,1,1,1,9.8), color_1st_mass='b', color_2nd_mass='r'):

    l1, l2, m1, m2, g = args

    theta1 = sol[:k,0]
    theta2 = sol[:k,1]

    # Plotting angles
    axF.plot(t[:k],theta1,'-', c=color_1st_mass)
    axF.plot(t[:k],theta2,'-', c=color_2nd_mass)
    axF.grid(True)
    axF.set_title("Time t= %.2f" % (t[k]), fontsize=20)
    axF.set_xlim([0, T])

    # Plotting Trajectories
    x1     = l1*np.sin(theta1)
    y1     = -l1*np.cos(theta1)
    x2     = x1+l2*np.sin(theta2)
    y2     = y1-l2*np.cos(theta2)
    axT.plot(x1,y1,'-o', alpha=0.05, c=color_1st_mass)
    axT.plot(x2,y2,'-o', alpha=0.05, c=color_2nd_mass)

    # Plotting masses Trajectories
    theta1 = sol[k,0]
    theta2 = sol[k,1]
    x1     = l1*np.sin(theta1)
    y1     = -l1*np.cos(theta1)
    x2     = x1+l2*np.sin(theta2)
    y2     = y1-l2*np.cos(theta2)
    axT.plot([0, x1],[0, y1],'-k')
    axT.plot([x1, x2],[y1, y2],'-k')
    axT.scatter(x1, y1, s=200, marker='o', c=color_1st_mass)
    axT.scatter(x2, y2, s=200, marker='o', c=color_2nd_mass)

    # Extra plottting configurations
    axT.grid(True)
    axT.set_title("Time t= %.2f" % (t[k]), fontsize=20)
    axT.set_xlim([-2.5,2.5])
    axT.set_ylim([-2.5,2.5])


I am working on the file "SC1v2/Bonus - 11 - Pendulum, double pendulum and chaos.ipynb".
The first task is:
The equations will be omitted but we refer to the interested reader to look at https://demonstrations.wolfram.com/DoublePendulum/ and https://scienceworld.wolfram.com/physics/DoublePendulum.html.
Can you write Python code for it?

def doublePendulum(y, t, l1, l2, m1, m2, g):
    th1  = y[0]
    th2  = y[1]
    th1p = y[2]
    th2p = y[3]
    th1pp = -((g*(2*m1 + m2)*np.sin(th1) + g*m2*np.sin(th1 - 2*th2) +
               2*m2*(l2*th2p**2 + l1*th1p**2*np.cos(th1 - th2))*np.sin(th1 - th2))
              /(2*l1*(m1 + m2 - m2*np.cos(th1 - th2)**2)))
    th2pp = (((m1 + m2)*(l1*th1p**2 + g*np.cos(th1))+
              l2*m2*th2p**2*np.cos(th1 - th2))*
             np.sin(th1 - th2))/(l2*(m1 + m2 - m2*np.cos(th1 - th2)**2))
    return np.array([th1p,th2p,th1pp,th2pp])

def plotDoblePendulum(k=0, T=10, N=1000, th10=np.pi, ome10=2, th20=-np.pi/2, ome20=-2):
    l1 = 1
    l2 = 1
    m1 = 1
    m2 = 1
    g  = 9.8

    t  = np.linspace(0,T,N)

    y0 = np.array([th10, th20, ome10, ome20])
    sol = odeint(doublePendulum, y0, t, args=(l1, l2, m1, m2, g))

    # Plotting trajectories
    theta1 = sol[:k,0]
    theta2 = sol[:k,1]
    x1     = l1*np.sin(theta1)
    y1     = -l1*np.cos(theta1)
    x2     = x1+l2*np.sin(theta2)
    y2     = y1-l2*np.cos(theta2)

    plt.figure(figsize=(16,8))

    ax1 = plt.subplot(121)
    ax2 = plt.subplot(122)

    ax1.plot(x1,y1,'-ob', alpha=0.05)
    ax1.plot(x2,y2,'-or', alpha=0.05)

    # Plotting function values
    ax2.plot(t[:k], theta1, 'b', label=r'$\theta_1$(t)')
    ax2.plot(t[:k], theta2, 'r', label=r'$\theta_2$(t)')
    ax2.legend(loc='lower right',fontsize=20)
    ax2.set_xlabel(r'$t$', fontsize=20)
    ax2.set_xlim([0,t[-1]])
    ax2.grid(True)

    # Plotting masses
    theta1 = sol[k,0]
    theta2 = sol[k,1]
    x1     = l1*np.sin(theta1)
    y1     = -l1*np.cos(theta1)
    x2     = x1+l2*np.sin(theta2)
    y2     = y1-l2*np.cos(theta2)
    ax1.plot([0, x1],[0, y1],'-k')
    ax1.plot([x1, x2],[y1, y2],'-k')
    ax1.scatter(x1, y1, s=200, marker='o', c='b')
    ax1.scatter(x2, y2, s=200, marker='o', c='r')

    # Extra plottting configurations
    ax1.grid(True)
    ax1.set_title("Time t= %.2f" % (t[k]), fontsize=20)
    #plt.axis('equal')
   ...


I am working on the file "SC2/U1_EigenWorld.ipynb".
The first task is:
<div id='invpi' />

Inverse Power Iteration
Una de las complicaciones que tiene el algoritmo anterior, es que sólo permite encontrar el valor y vectores propios dominantes. Luego ¿Cómo encontramos el resto?. Para responder esta pregunta, es necesario examinar dos propiedades importantes:

Los valores propios de la matriz inversa $A^{-1}$ son los recíprocos de los valores propios de $A$, es decir: $\lambda_1^{-1}, \lambda_2^{-1}, \ldots , \lambda_m^{-1}$. Los vectores propios de se mantienen inalterados.
Los valores propios de la matriz con shift $A - sI$ son: $\lambda_1-s, \lambda_2-s, \ldots, \lambda_m-s$. Del mismo modo, los vectores propios se mantienen inalterados.

Tarea: Pruebe estas propiedades!
La idea es entonces realizar un shift $\widetilde{s}$ cercano a algún valor propio $s_k$, y computar PI sobre $(A - \widetilde{s}I)^{-1}$. Luego:
$$ |\lambda_k - \widetilde{s}| < |\lambda_i - \widetilde{s}| \leftrightarrow  \bigg| \frac{1}{\lambda_k - \widetilde{s}} \bigg| > \bigg| \frac{1}{\lambda_i - \widetilde{s}} \bigg| \ \ \forall i \neq k \ $$
entonces $\frac{1}{\lambda_k - \widetilde{s}}$ corresponderá con el vector propio dominante de $(A - \widetilde{s}\,I)^{-1}$. Notar que por lo enunciado en las propiedades, los vectores propios se mantienen sin alteraciones.
La idea anterior se ve reflejada en el algoritmo implementado a continuación:
Can you write Python code for it?

def inverse_power_iteration(A, x, s, k, verbose=False):
    """
    Program 12.2 Inverse Power iteration
    Computes eigenvector of square matrix nearest to input s
    Input: matrix A, initial (nonzero) vector x, shift s, number of steps k
    Output: dominant eigenvalue lam, eigenvector of inv(A-sI)
    """
    if verbose: print("Inverse Power Iteration Method\n%s"%('='*80))
    As = A - s*np.eye(*A.shape)
    for j in range(k):
        u = x/np.linalg.norm(x)
        x = np.linalg.solve(As, u) # Critical line!
        lam = np.dot(u.T, x)
        if verbose: print("k=%d, lambda=%+.3f, u=%s"%(j,1./lam+s,str(u.T)))
    u = x/np.linalg.norm(x)
    if verbose: print("k=%d, lambda=%+.3f, u=%s\n"%(j+1,1./lam+s,str(u.T)))
    return (1./lam+s, u)

# Testing algoritm
lam, u = inverse_power_iteration(A, x, s=1./4, k=10, verbose=True)
print("lambda = {0}".format(lam))
print("v = {0}".format(u))


I am working on the file "SC2/U1_EigenWorld.ipynb".
The first task is:
<div id='sp' />

$\texttt{SciPy}$ Eigenvalue
La librería scipy tiene implementados algoritmos que permite calcular los valores y vectores propios. Las opciones posibles son:

En la librería scipy.linalg: eigvals/eigvalsh/eigvals_banded, eig/eigh/eig_banded,

En la librería scipy.sparse.linalg: eigen, eigs, eigsh.

En general siempre conviene utilizar las funciones desde scipy y no de numpy. La librería numpy hace un excelente trabajo al permitir el uso de vectores de tipo numérico, pero contiene solo algunos algoritmos numéricos y no necesariamente los más rápidos.
A continuación se muestra como utilizar algunas de estas funciones.
Can you write Python code for it?

# Full matrices
from scipy import linalg as LA
N = 3
Aux = np.random.rand(N,N)
A = Aux + Aux.T # symmetric, so we'll deal with real eigs.
print(LA.eigvals(A)) # Only the eigenvalues, A not necessarily symmetric
print("*"*80)
print(LA.eigvalsh(A)) # Only the eigenvalues, A symmetric
print("*"*80)
print(LA.eig(A))     # All the eigenvalues and eigenvectors, A not necessarily symmetric
print("*"*80)
print(LA.eigh(A))    # All the eigenvalues and eigenvectors, A symmetric (faster)
print("*"*80)
lambdas, V = LA.eigh(A)    # All the eigenvalues and eigenvectors, A symmetric (faster)
l1 = lambdas[0]
v1 = V[:,0]
print(l1)
print(v1)
print(np.dot(A, v1))
print(l1*v1)


I am working on the file "SC2/U1_EigenWorld.ipynb".
The first task is:
Matriz y vector de prueba
Can you write Python code for it?


I am working on the file "SC2/U1_EigenWorld.ipynb".
The first task is:
Preguntas:
1. ¿Porque es necesario el try y except en las líneas 11 y 13? ¿Que significa que el sistema no pueda ser resuelto?
2. Como puede observar RQI no recibe shift como parámetro. ¿A cuál valor/vector propio convergerá? ¿Como forzar/guiar a que tienda hacia un valor/vector propio distinto?
Can you write Python code for it?

# Testing algorithm
lam, v = rqi(A, x, k=2)
print("lambda = {0}".format(lam))
print("v = {0}".format(v))


I am working on the file "SC2/U2_QuadWorldAll.ipynb".
The first task is:
<div id='cg' />

Cuadratura Gaussiana
Similar a las ideas que propuso Chebyshev para mejorar los método de interpolación, nace la siguiente pregunta:

¿Se podrá mejorar el proceso de integración particionando $[a,b]$ de una forma no equiespaciada?

La respuesta a esta pregunta es sí. De modo análogo a los puntos de Chebyshev, aca podemos seleccionar las raíces de los polinomios de Legendre $p_n(x)$ siguientes:
\begin{align}
p_n(x) = \frac{1}{2^n n!} \frac{d^n}{dx^n} \left[ (x^2 - 1)^n \right]
\end{align}
para luego interpolar $f$ sobre estos puntos, generando un polinomio $p_{n-1}$ sobre el cual realizar la integración.
Este método utiliza la siguiente aproximación, para un intervalo $[-1,1]$:
\begin{align}
\int_{a}^{b} f(x) dx \approx \sum_{i=1}^n w_i f(x_i)
\end{align}
donde los $x_i$ se definen como las raíces del n-ésimo polinomio de Legendre $p_n(x)$, y los $w_i$ se calculan como:
$$
w_i = \int_{-1}^{1}L_{i}(x)dx, \ \ \  i = 1,\dots,n
$$
siendo $L_i(x)$ los conocidos polinomio de la interpolación de Lagrange.
Nota: Para un intervalo arbitrario $[a,b]$ es necesario realizar la transformación lineal correspondiente.
Can you write Python code for it?

def gaussianquad(myfun, N, a, b, verbose=False, text="", figname=""):
    f = np.vectorize(myfun) # So we can apply it to arrays without trouble
    x, w = gaussian_nodes_and_weights(N, a, b)
    int_val = sum( w * f(x) )
    if verbose:
        xbin, ybin = gaussian_bins(f, x, w)
        plot(f, xbin, ybin, int_val, N, text, figname)
    return int_val

# Comment: These nodes could be precomputed in advance, here
# they are computed as needed so it will penalize the
# computation time.
def gaussian_nodes_and_weights(N, a, b):
    if N==1:
        return np.array([1]), np.array([2])
    beta = .5 / np.sqrt(1.-(2.*np.arange(1.,N))**(-2))
    T = np.diag(beta,1) + np.diag(beta,-1)
    D, V = np.linalg.eigh(T)
    x = D
    x = .5 * ( (b-a)*x + b + a) # Rescaling
    w = 2*V[0,:]**2
    w = .5*(b-a)*w
    return x, w

def gaussian_bins(f, x, w):
    z = [a] + list(a + w.cumsum())
    xbin = np.vstack([z[:-1], z[1:]]).flatten('F')
    z = f(x)
    ybin = np.vstack([z[:], z[:]]).flatten('F')
    return np.array(xbin), np.array(ybin)

print('Approximated sum: {0}'.format(gaussianquad(myfun, N, a, b, verbose=True,
                                    text=text, figname="gaussianquad_%d.png"%N)))

gaussian_nodes_and_weights(2, -1, 1)


I am working on the file "SC2/U2_QuadWorldAll.ipynb".
The first task is:
<div id='mp' />

Midpoint
Una de las limitaciones de las dos reglas anteriores, es que requieren evaluar $f$ en los extremos del intervalo, y pueden haber casos en donde $f$ no esté bien definida en tales puntos.
La regla del punto medio midpoint para cada dos puntos $(x_k,f(x_k))$ y $(x_{k+1},f(x_{k+1}))$, aproxima la función por una expansión de Taylor de grado $1$, centrada en el punto medio $\displaystyle \left( \frac{x_k+x_{k+1}}{2}, f\left(\frac{x_k+x_{k+1}}{2} \right) \right)$, es decir, aproxima $f$ por un polinomio de grado $1$ (al igual que la regla del trapecio).
Al particionar el intervalo $[a,b]$ en $m$ segmentos y $m+1$ puntos $a = x_0 < \cdots < x_{m} = b \ \ $, se obtiene el siguiente resultado:
\begin{align}
\int_{a}^{b} f(x) dx = \sum_{i=1}^{m} \int_{x_{i-1}}^{x_{i}} f(x) dx  \
                     = \sum_{i=1}^{m} h f(w_i) + \underbrace{\frac{(b-a)}{24} h^2 f''(c)}_{\text{Error term}}
\end{align}
donde $h=(b-a)/m \ \ \ $,
$w_i=\frac{1}{2}(x_{i-1}+x_{i})\ \ \ $
y
$\ \ c \in [a, b]$
Can you write Python code for it?

def midpoint(myfun, N, a, b, verbose=False, text='', figname=''):
    f = np.vectorize(myfun) # So we can apply it to arrays without trouble
    x = np.linspace(a, b, N+1) # We want N bins, so N+1 points
    dx = x[1]-x[0]
    midpoints = x[:-1] + .5*dx
    midpoint_values = f(midpoints)
    int_val = sum(midpoint_values*dx)
    if verbose:
        xbin = np.vstack([x[:-1], x[1:]]).flatten('F')
        ybin = np.vstack([midpoint_values, midpoint_values]).flatten('F')
        plot(f, xbin, ybin, int_val, N, text, figname)
    return int_val

N = 20
#myfun = lambda x : x + 1
print('Approximated sum: {0}'.format(midpoint(myfun, N, a, b, verbose=True, text=text, figname="midpoint_%d.png"%N)))


I am working on the file "SC2/U2_QuadWorldAll.ipynb".
The first task is:

Métodos de Newton-Cotes
Los siguientes dos métodos que se presentan a continuación, conforman parte de una familia de métodos llamados de Newton-Cotes. Estos generan una malla equiespaciada para particionar el dominio de integración $[a,b]$.
Todos estos métodos se basan en aproximar la función $f$ por un polinomio de grado $n-1: \ p_{n-1}$, que interpole $n$ puntos de tal malla, y de este modo computar el área bajo la curva de este polinomio (En vez de $f$), lo cual es fácil dado que las integrales de polinomios son también polinomios (de un grado superior).
Las derivaciones de las fórmulas a utilizar pueden ser consultadas en el texto guía: Numerical Analysis, Timothy Sauer.

Regla del Trapecio
El primero y más simple de tales métodos, es utilizar polinomios de grado $1$ que interpolen cada $(x_k,f(x_k))$ y $(x_{k+1},f(x_{k+1}))$. Es fácil notar que como resultado, el área aproximada entre cada dos puntos de la malla, será el área de un trapecio, motivo del nombre de tal regla.
Al particionar el intervalo $[a,b]$ en $m$ segmentos y $m+1$ puntos $a = x_0 < \cdots < x_{m} = b \ \ $, se obtiene el siguiente resultado
\begin{align}
\int_{x_0}^{x_m} f(x) dx = \sum_{i=1}^{m} \int_{x_{i-1}}^{x_{i}} f(x) dx
                         = \frac{h}{2}\left[f(a) + f(b) + 2\sum_{i=1}^{m-1} f(x_i) \right] - \underbrace{(b-a) \frac{h^2}{12} f''(c)}_{\text{Error term}}
\end{align}
donde $h=(b-a)/m \ $ es el largo de cada subintervalo, y $\ c  \in [a, b]$. Dado que $c$ no es conocido, el Error term en la práctica no se toma en cuenta, y por lo mismo constituye el error del método.
Can you write Python code for it?

def trapezoid(myfun, N, a, b, verbose=False, text='', figname=''):
    f = np.vectorize(myfun) # So we can apply it to arrays without trouble
    x = np.linspace(a, b, N+1) # We want N bins, so N+1 points
    h = x[1]-x[0]
    xleft = x[:-1]
    xright = x[1:]
    int_val = 0.5*h*sum(f(xleft)+f(xright))
    if verbose:
        xbin = x
        ybin = f(x)
        plot(f, xbin, ybin, int_val, N, text, figname)
    return int_val

N = 40
#myfun = lambda x : x**2
print('Approximated sum: {0}'.format(trapezoid(myfun, N, a, b, verbose=True, text=text, figname="trapezoid_%d.png"%N)))


I am working on the file "SC2/U2_QuadWorldAll.ipynb".
The first task is:
Parámetros
Para los experimentos numéricos, consideraremos la función $f(x)$ con dominio $x \in [a,b]$. El número de puntos a utilizar en las mallas equiespaciadas vendrá definido por $N$. Todas las anteriores se definen en la celda siguiente.
Can you write Python code for it?

#limits of integration
a = -1; b = 1
#function to integrate
myfun = lambda x : np.exp(x) #x**2 #1 # x #np.exp(-x)
#number the points in the 1D grid
N = 10
#text to show in the graphs
text= r"\int_{%+.2f}^{%+.2f} e^x dx" %(a,b)