# Math for Non-Geeks/ Derivatives of higher order
## Motivation

The derivative $f'$ describes the current rate of change of the function $f$. Now the derivative function $f'$ can be differentiated again, provided that it is again differentiable. The obtained derivative of the derivative is called second derivative or derivative of second order and is called $f''$ or $f^{(2)}$. This can be done arbitrarily often. If the second derivative is again differentiable, a third derivative $f^{(3)}$ can be constructed, then a fourth derivative $f^{(4)}$ and so on.
These higher derivatives allow statements about the course of a function graph. The second derivative tells us whether a graph is curved upwards ("convex") or curved downwards ("concave"). If a function has a convex graph, its gradient increases continuously. For this convexity, $f''(x)>0$ is a sufficient condition. If the second derivative is always positive, then the first derivative must grow continuously. Analogously, it follows from $f''(x)<0$ that the graph is concave and the derivative falls monotonically.
Higher-order derivatives do not only tell us more about abstract functions, they can also have a physical meaning. Consider the function $f:[a,b]\to \mathbb {R}$ with $f(t)=t^{3}+2$, which shall describe the location $f(t)$ of a car at the time $t$. We already know that we can calculate the speed of the car at the time $t$ with the first derivative: $f'(t)=f^{(1)}(t)=3t^{2}$. What does the derivative $f''(t)$ of $f'$ say? This is the instantaneous rate of change of speed and thus the acceleration of the car. It accelerates with $f''(t)=f^{(2)}(t)=6t$. So second derivatives describe accelerations.
Now we can derive this second derivative again, whereby we get the rate of change of acceleration $f'''(t)=f^{(3)}(t)=6$. This is called jerk in vehicle dynamics and indicates how fast a car increases acceleration or how fast it initiates braking. For example, a big jerk occurs during emergency braking. Since $f^{(3)}(t)<0$ is in an emergency stop, the graph of the speed $f'$ is convex - the speed decreases more and more. The fourth derivative $f^{4}(t)=0$ again tells us that the jerk has no instantaneous rate of change.

## Definition

Definition (Derivatives of higher order)
Let $f:D\to W$ with $D,W\subseteq \mathbb {R}$ be a real function. We set $f^{(0)}:=f$ and in the case of differentiability $f^{(1)}=f'$. We define the second derivative via $f^{(2)}=\left(f^{(1)}\right)'$, the third derivative via $f^{(3)}=\left(f^{(2)}\right)'$ etc., if these higher derivatives exist. Overall, we define recursively for $k\in \mathbb {N} _{0}$:
$f^{(k+1)}:=\left(f^{(k)}\right)'$
We say $f$ that is $k$ times differentiable, if the $k$-th derivative $f^{(k)}$ of $f$ exists. $f$ is called $k$ times continuously differentiable, if $f^{(k)}$ is continuous (which is a stronger statement).
The set of all $k$ times continuously differential functions with domain of definition $D$ and range $W$ is denoted $C^{k}(D,W)$. In particular $C^{0}(D,W)=C(D,W)$ consists of the continuous functions. If we can derive the function $f$ arbitrarily often, we write $f\in C^{\infty }(D,W)$. If $W=\mathbb {R}$, then we can write $C^{k}(D)$ or $C^{\infty }(D)$ in short. Those sets of functions satisfy the inclusion chain:
$C(D,W)\supseteq C^{1}(D,W)\supseteq C^{2}(D,W)\supseteq \ldots \supseteq C^{k}(D,W)\supseteq C^{k+1}(D,W)$
Question: Are the following statements true of false?
* $|x|\in C(\mathbb {R} )$
* $|x|\in C^{1}(\mathbb {R} )$
* $\operatorname {sgn}(x)\in C(\mathbb {R} )$
* $\operatorname {sgn}(x)\in C^{1}(\mathbb {R} )$
* $x\in C(\mathbb {R} )$
* $x\in C^{1}(\mathbb {R} )$
Solutions:
* true
* false
* false
* false
* true
* true

## Examples for higher derivatives
### Derivatives of the power function

Example (Derivatives of the power function)
We consider the function $f:\mathbb {R} \to \mathbb {R} ,x\mapsto x^{2}$. This function is infinitely often differentiable, since there is for all $x\in \mathbb {R}$ and all $k\in \mathbb {N} ,k\geq 3$:
${\begin{aligned}&\ f'(x)=2x\\[0.3em]&\ f''(x)=2\\[0.3em]&\ f^{(k)}=0\\[0.3em]\end{aligned}}$
In general, for $f:\mathbb {R} \to \mathbb {R} ,x\mapsto x^{n}$ with $n\in \mathbb {N}$ there is:
${\begin{aligned}&\ f'(x)=nx^{n-1}\\[0.3em]&\ f^{(k)}(x)=n(n-1)\cdot \ldots \cdot (n-k+1)x^{n-k}{\text{ for }}k\leq n\\[0.3em]&\ f^{(k)}=0{\text{ for }}k>n\\[0.3em]\end{aligned}}$

### Derivatives of the exponential function

Example (Derivatives of the exponential function)
For the exponential function $\exp :\mathbb {R} \to \mathbb {R}$ since $\exp '(x)=\exp(x)$ for all $x\in \mathbb {R}$ we have infinite differentiability $\exp \in C^{\infty }(\mathbb {R} ,\mathbb {R} )$. In addition there is for all $k\in \mathbb {N}$:
$\exp ^{(k)}(x)=\exp(x)$

### Derivatives of the sine function

Example (Derivatives of the sine function)
The function $\sin :\mathbb {R} \to \mathbb {R}$ is infinitely often continuously differentiable. For all $x\in \mathbb {R}$ there is:
${\begin{aligned}&\sin '(x)=\cos(x)\\[0.3em]&\sin ''(x)=\cos '(x)=-\sin(x)\\[0.3em]&\sin '''(x)=(-\sin )'(x)=-\cos(x)\\[0.3em]&\sin ^{(4)}(x)=(-\cos )'(x)=\sin(x)\\[0.3em]\end{aligned}}$
In general, for all $n\in \mathbb {N} _{0}$ there is:
${\begin{aligned}&\sin ^{(4n+1)}(x)=\cos(x)\\[0.3em]&\sin ^{(4n+2)}(x)=\cos '(x)=-\sin(x)\\[0.3em]&\sin ^{(4n+3)}(x)=(-\sin )'(x)=-\cos(x)\\[0.3em]&\sin ^{(4n)}(x)=(-\cos )'(x)=\sin(x)\\[0.3em]\end{aligned}}$
Question: What are the derivatives von $\cos :\mathbb {R} \to \mathbb {R}$?
We use that $\sin '(x)=\cos(x)$. For $x\in \mathbb {R}$ there is
${\begin{aligned}&\cos '(x)=-\sin(x)\\[0.3em]&\cos ''(x)=-\cos(x)\\[0.3em]&\cos '''(x)=\sin(x)\\[0.3em]&\cos ^{(4)}(x)=\cos(x)\end{aligned}}$
In general, for all $n\in \mathbb {N} _{0}$ there is:
${\begin{aligned}&\cos ^{(4n+1)}(x)=-\sin(x)\\[0.3em]&\cos ^{(4n+2)}(x)=-\cos(x)\\[0.3em]&\cos ^{(4n+3)}(x)=\sin(x)\\[0.3em]&\cos ^{(4n)}(x)=\cos(x)\end{aligned}}$

## Exercises: higher derivatives
### Derivatives of the logarithm function

Math for Non-Geeks: Template:Aufgabe

### Exactly once differentiable function

Math for Non-Geeks: Template:Aufgabe
Additional question: Is $f'$ continuous at $0$?
Nope. Take the two sequences:
${\begin{aligned}&\ (a_{n})_{n\in \mathbb {N} }=({\tfrac {1}{2n\pi }})_{n\in \mathbb {N} }\\[0.3em]&\ (b_{n})_{n\in \mathbb {N} }=({\tfrac {1}{(2n+1)\pi }})_{n\in \mathbb {N} }\\[0.3em]\end{aligned}}$
For these sequences, there is: $\lim _{n\to \infty }a_{n}=\lim _{n\to \infty }b_{n}=0$. However
${\begin{aligned}&\ f'(a_{n})=2{\frac {1}{2n\pi }}\cdot \underbrace {\sin \left(2n\pi \right)} _{=0}-\underbrace {\cos \left(2n\pi \right)} _{=1}=-1\\[0.3em]&\ f'(b_{n})=2{\frac {1}{(2n+1)\pi }}\cdot \underbrace {\sin \left((2n+1)\pi \right)} _{=0}-\underbrace {\cos \left((2n+1)\pi \right)} _{=-1}=1\end{aligned}}$
So $\lim _{x\to 0}f'(x)$ doesn't exist. By means of the sequence criterion, $f'$ is hence not continuous at $0$.
Remark: Therefore, $f'$ is also not differentiable at $0$.

## Computation rules for higher derivatives
### Linearity

The linearity of derivatives is also "inherited" to higher derivatives: If $f$ and $g$ are differentiable, for $a,b\in \mathbb {R}$ the function $af+bg$ is also differentiable with
$(af+bg)'=af'+bg'$
If $f$ and $g$ are now even twice differentiable, then there is
$(af+bg)''=(af'+bg')'=af''+bg''$
If we continue to do so, we will get
Theorem (Linearity of higher derivatives)
Let $a,b\in \mathbb {R}$ and $f,g:D\to \mathbb {R}$ be $n$ times differentiable. Then also $af+bg:D\to \mathbb {R}$ is $n$ times differentiable, and for all $n\in \mathbb {N} _{0}$ there is:
$(af+bg)^{(n)}=af^{(n)}+bg^{(n)}$
Example (Linearity of higher derivatives)
Since $\sin ^{(4n+1)}=\cos$ and $\cos ^{(4n+1)}=-\sin$ for $n\in \mathbb {N}$ there is
$(3\sin(x)+4\cos(x))^{(1001)}=3\sin ^{(1001)}(x)+4\cos ^{(1001)}(x)=3\sin ^{(4\cdot 250+1)}(x)+4\cos ^{(4\cdot 250+1)}(x)=3\cos(x)-4\sin(x)$
Proof (Linearity of higher derivatives)
Theorem whose validity shall be proven for $n\in \mathbb {N}$ bewiesen werden soll:
$(af+bg)^{(n)}=af^{(n)}+bg^{(n)}$
1. Base case:
$(af+bg)^{(0)}=af+bg=af^{(0)}+bg^{(0)}$
2. Inductive step:
2a. Inductive hypothesis:
$(af+bg)^{(n)}=af^{(n)}+bg^{(n)}$
2b. Induction theorem:
$(af+bg)^{(n+1)}=af^{(n+1)}+bg^{(n+1)}$
2c. Proof of induction step:
${\begin{aligned}(af+bg)^{(n+1)}&=((af+bg)^{(n)})'\\[0.3em]&{\color {OliveGreen}\left\downarrow \ {\text{induction assumption}}\right.}\\[0.3em]&=(af^{(n)}+bg^{(n)})'\\[0.3em]&{\color {OliveGreen}\left\downarrow \ {\text{linearity of the derivative}}\right.}\\[0.3em]&=af^{(n+1)}+bg^{(n+1)}\end{aligned}}$

### Leibniz rule for product functions

We now try to determine a general formula for the $n$-th derivative of the product function $fg$ of two arbitrarily often differentiable functions $f$ and $g$. By applying the factor-, sum- and product rule several times we obtain for $n=1,2,3$
${\begin{aligned}(fg)'&=f'g+fg'\\(fg)''&=(f'g+g'f)'=(f'g)'+(fg')'\\&=f''g+f'g'+f'g'+fg''\\&=f''g+2f'g'+fg''\\(fg)'''&=(f''g+2f'g'+fg'')'\\&=(f''g)'+(2f'g')'+(fg'')'\\&=f'''g+f''g'+2f''g'+2f'g''+f'g''+fg'''\\&=f'''g+3f''g'+3f'g''+fg'''\end{aligned}}$
If we plug in $f=x$ and $g=y$, and instead of the derivatives of $f$ and $g$ the corresponding powers of $x$ and $y$, we see a clear analogy to the binomial theorem:
${\begin{aligned}(x+y)^{1}&=x^{1}y^{0}+x^{0}+y^{1}\\(x+y)^{2}&=x^{2}+2xy+y^{2}\\&=x^{2}y^{0}+2x^{1}y^{1}+x^{0}y^{2}\\(x+y)^{3}&=x^{3}+3x^{2}y+3xy^{2}+y^{3}\\&=x^{3}y^{0}+3x^{2}y^{1}+3x^{1}y^{2}+3x^{0}y^{3}\end{aligned}}$
This analogy can be made clear as follows:
We assign for every $k\in \mathbb {N} _{0}$ the derivative $f^{(k)}$ to the power $x^{k}$, and the derivative $g^{(k)}$ to the power $y^{k}$. The $0$-th derivative $f^{(0)}=f$ corresponds to the $0$-th power $x^{0}=1$. The derivative of the term $f^{(k)}g^{(l)}$ is by means of the product rule
$(f^{(k)}g^{(l)})'=f^{(k+1)}g^{(l)}+f^{(k)}g^{(l+1)}$
The expression $f^{(k+1)}g^{(l)}+f^{(k)}g^{(l+1)}$ now corresponds in our analogy to the sum $x^{k+1}y^{l}+x^{k}y^{l+1}$. We get this term from $x^{k}y^{l}$ by multiplication with $x+y$. For our polynomials, the distributive law yields
$(x^{k}y^{l})(x+y)=x^{k+1}y^{l}+x^{k}y^{l+1}$
Therefore, the application of the product rule corresponds to the multiplication with the sum $x+y$. Thus the $n$-th derivative $(fg)^{(n)}$ corresponds to the power $(x+y)^{n}$. From the binomial theorem
$(x+y)^{n}=\sum _{k=0}^{n}{\binom {n}{k}}x^{k}y^{n-k}$
we hence get the
Theorem (Leibniz rule for derivatives)
Let $f,g:D\to \mathbb {R}$ be $n$ times differentiable functions. Then, $fg:D\to \mathbb {R}$ is $n$ times differentiable, and for all $n\in \mathbb {N} _{0}$, there is:
$(fg)^{(n)}=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}$
Example (Leibniz rule for derivatives)
Using the Leibniz rule we calculate $(x^{3}e^{x})^{(2016)}$. The rule is applicable because $x\mapsto x^{3}$ and $x\mapsto e^{x}$ are arbitrarily often differentiable on $\mathbb {R}$. There is
${\begin{aligned}(x^{3}e^{x})^{(2016)}&{\underset {\text{rule}}{\overset {\text{Leibniz}}{=}}}\sum _{k=0}^{2016}{\binom {2016}{k}}(x^{3})^{(k)}(e^{x})^{(2016-k)}\\[0.3em]&\color {Gray}\left\downarrow \ (x^{3})'=3x^{2},\ (x^{3})''=6x,\ (x^{3})^{(3)}=6,\ (x^{3})^{(k)}=0{\text{ for }}k\geq 4,\ (e^{x})^{(k)}=e^{x}{\text{ for all }}k\in \mathbb {N} \right.\\[0.3em]&={\binom {2016}{0}}x^{3}e^{x}+{\binom {2016}{1}}\cdot 3x^{2}e^{x}+{\binom {2016}{2}}\cdot 6xe^{x}+{\binom {2016}{3}}\cdot 6e^{x}\\[0.3em]&=x^{3}e^{x}+2016\cdot 3x^{2}e^{x}+{\frac {2016\cdot 2015}{2}}\cdot 6xe^{x}+{\frac {2016\cdot 2015\cdot 2014}{3\cdot 2}}\cdot 6e^{x}\\[0.3em]&=x^{3}e^{x}+2016\cdot 3x^{2}e^{x}+2016\cdot 2015\cdot 3xe^{x}+2016\cdot 2015\cdot 2014e^{x}\end{aligned}}$
Proof (Leibniz rule for derivatives)
Theorem whose validity shall be proven for $n\in \mathbb {N}$ bewiesen werden soll:
$(fg)^{(n)}=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}$
1. Base case:
$(fg)^{(0)}=fg=\sum _{k=0}^{0}{\binom {0}{k}}f^{(k)}g^{(0-k)}$
2. Inductive step:
2a. Inductive hypothesis:
$(fg)^{(n)}=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}$
2b. Induction theorem:
$(fg)^{(n+1)}=\sum _{k=0}^{n+1}{\binom {n+1}{k}}f^{(k)}g^{(n+1-k)}$
2c. Proof of induction step:
${\begin{aligned}(fg)^{(n+1)}&=\left[(fg)^{(n)}\right]'\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ induction assumption}}\right.\\[0.3em]&=\left[\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k)}\right]'\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ linearity of the derivative}}\right.\\[0.3em]&=\sum _{k=0}^{n}{\binom {n}{k}}\left[f^{(k)}g^{(n-k)}\right]'\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ product rule}}\right.\\[0.3em]&=\sum _{k=0}^{n}{\binom {n}{k}}\left[f^{(k+1)}g^{(n-k)}+f^{(k)}g^{(n-k+1)}\right]\\[0.3em]&=\sum _{k=0}^{n}{\binom {n}{k}}f^{(k+1)}g^{(n-k)}+\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k+1)}=\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ index shift}}\right.\\[0.3em]&=\sum _{k=1}^{n+1}{\binom {n}{k-1}}f^{(k)}g^{(n-(k-1))}+\sum _{k=0}^{n}{\binom {n}{k}}f^{(k)}g^{(n-k+1)}\\[0.3em]&\color {Gray}\left\downarrow \ {\binom {n}{-1}}=0{\text{ and }}{\binom {n}{n+1}}=0\right.\\[0.3em]&=\sum _{k=0}^{n+1}{\binom {n}{k-1}}f^{(k)}g^{(n-k+1)}+\sum _{k=0}^{n+1}{\binom {n}{k}}f^{(k)}g^{(n-k+1)}\\[0.3em]&=\sum _{k=0}^{n+1}\left[{\binom {n}{k-1}}+{\binom {n}{k}}\right]f^{(k)}g^{(n-k+1)}\\[0.3em]&\color {Gray}\left\downarrow \ {\text{ sum of binomial coefficients}}\right.\\[0.3em]&=\sum _{k=0}^{n+1}{\binom {n+1}{k}}f^{(k)}g^{((n+1)-k)}\end{aligned}}$