# Explaining a Math Magic Trick

Introduction
------------

A recent popular tweet did a "math magic trick", and I want to explain why it works and use that as an excuse to talk about cool math (functional analysis). The [tweet in question](https://twitter.com/GuilleAngeris/status/1784936202686300283):

![Image](https://pbs.twimg.com/media/GMVd9XUacAA33yU?format=jpg&name=medium)

This is a cute magic trick, and like any good trick they nonchalantly gloss over the most important step. Did you spot it? Did you notice your confusion?

Here's the key question: **Why did they switch from a differential equation to an integral equation?** If you can use $(1-x)^{-1}=1+x+x^2+...$ when $x=\int$, why not use it when $x=d/dx$? 

Well, lets try it, writing $D$ for the derivative:

$\begin{eqnarray} f'&=&f\ (1-D)f&=&0\ f&=&(1+D+D^2+...)0\ f&=&0+0+0+...\ f&=&0 \end{eqnarray}$

So now you may be disappointed, but relieved: yes, this version fails, but at least it fails-safe, giving you the trivial solution, right?

But no, **actually **$(1-D)^{-1}=1+D+D^2+...$** can fail catastrophically,** which we can see if we try a nonhomogeneous equation like $f'=f+e^x$ (which you may recall has solution $xe^x$):

$\begin{eqnarray}f'&=&f+e^x\(1-D)f&=&e^x\\f&=&(1+D+D^2+...)e^x\\f&=&e^x+e^x+e^x+...\\f&=&\infty?\end{eqnarray}$

However, **the integral version still works.** To formalize the original approach: we define the function $I$ (for integral) to take in a function $f(x)$ and produce the function $If$ defined by $If(x)=\int_0^xf(t)dt$. This rigorizes the original trick, elegantly incorporates the initial conditions of the differential equation, and fully generalizes to solving nonhomogeneous versions like $f'=f+e^x$ (left as an exercise to the reader, of course).

**So why does **$(1-D)^{-1}=1+D+D^2+...$** fail, but **$(1-I)^{-1}=1+I+I^2+...$**works robustly?** The answer is functional analysis!

Functional Analysis
-------------------

Savvy readers may already be screaming that the trick $(1-x)^{-1}=1+x+x^2+...$ for numbers only holds true for $|x|<1$, and this is indeed the key to explaining what happens with $D$ and $I$! But how can we define the "absolute value" of "the derivative function" or "the integral function"?

**What we're looking for is a** ***norm*****, a function that generalizes absolute values.** A norm is a function $x \mapsto ||x||$ satisfying these properties:

1.  $||x||\geq 0$ for all $x$ (*positivity*), and $||x||=0$ if and only if $x=0$ (*positive-definite*)
2.  $||x+y|| \leq ||x||+||y||$ for all $x$ and $y$ (*triangle inequality*)
3.  $||c x||= |c| \cdot ||x||$ for all $x$ and real numbers $c$, where $|c|$ denotes the usual absolute value (*absolute homogeneity*)

Here's an important example of a norm: fix some compact subset of $\mathbb R$, say $X=[-10, 10]$, and for a continuous function $f: X \rightarrow\mathbb{R}$ define $||f||_{\infty}=\max_{x \in X} |f(x)|$, which would commonly be called the $L^\infty$-norm of $f$. (We may use a maximum here due to the Extreme Value Theorem. In general you would use a supremum instead.) Again I shall leave it to the reader to check that this is a norm.

This example takes us halfway to our goal: **we can now talk about the "absolute value" of a continuous function** that takes in a real number and spits out a real number, but $D$ and $I$ take in functions and spit out functions (what we usually call an *operator,* so what we need is an *operator norm*).

**Put another way, the **$L^\infty$**-norm is "the largest output of the function"**, and this will serve as the inspiration for our operator norm. Doing the minimal changes possible, we might try to define $||I||=\max_{f \text{ continuous}}||If||_{\infty}$. There are two problems with this:

1.  First, since $I$ is linear, you can make $||If||_{\infty}$ arbitrarily large by scaling $f$ by 10x, or 100x, etc. We can fix this by restricting the set of valid *f* for these purposes, just like how for the $L^\infty$ example restricted the inputs of $f$ to the compact set $X=[-10,10]$. Unsurprisingly nice choice of set to restrict to is the "unit ball" of functions, the set of functions with $||f||_\infty \leq 1$.   
2.  Second, we must bid tearful farewell to the innocent childhood of maxima, and enter the liberating adulthood of suprema. This is necessary since $f$ ranges over the infinite-dimensional vector space of continuous functions, so the [Heine-Borel theorem](https://en.wikipedia.org/wiki/Heine%E2%80%93Borel_theorem) no longer guarantees the unit ball is compact, and therefore the extreme value theorem no longer guarantees that we will attain a maximum.

**So the proper definition of the norm of **$I$** and **$D$** are: **

$||I||=\sup_{f \text{ continuous}, ||f||_\infty \leq 1} ||If||_{\infty}$

$||D||=\sup_{f \text{ continuous}, ||f||_\infty \leq 1} ||Df||_{\infty}$

(and you can define similar norms for any linear operator, including $I^2$, etc.) A good exercise is to show these equivalent definitions of the operator norm for any linear function L:

$\begin{eqnarray}||L|| &=& \sup_{f \text{ continuous}, ||f||_\infty \leq 1} ||Lf||_{\infty}\ &=& \sup_{f \text{ continuous}, ||f||_\infty = 1} ||Lf||_{\infty} \ &=&\inf \{b \geq 0: ||Lf||_\infty \leq b||f||_\infty \text{ for all }f \} \end{eqnarray}$

So another way of thinking of the operator norm is the **maximum stretching factor of the linear operator.** The third definition also motivates the terminology of *bounded* linear operators: each such $b$ is a *bound* on the operator $L$, and the least such bound is the norm. Fun exercise: show that a linear operator is bounded if and only if it is continuous (with respect to the correct topologies). Hint: you'll need to work in infinite dimensional spaces here, because any finite-dimensional linear operator must be bounded.

**Now let's actually compute these norms!** For $I$, remember that our $L^\infty$-norm is defined over the interval $X=[-10,10]$. First observe that for the constant function $f(x)=1$, $If(x)=x$, so $||If||_{\infty}=10$. Thus $||I|| \geq 10$. To show that this is indeed the maximum we use the triangle inequality for integrals:

$\begin{eqnarray} ||If|| &=& \max_{x\in X} |\int_0^x f(t)dt|\&\leq & \max_{x\in X} \int_0^x |f(t)|dt\&\leq & \max_{x\in X} \int_0^x 1dt\&= & \max_{x\in X} x = 10\end{eqnarray}$

**So we have shown **$||I||=10$**!** Put a pin in that while we check $||D||$.

For $D$, we have a problem: for any positive number $k$, $D(e^{kx})=ke^{kx}$. In other words, $D$ can stretch functions by any amount, so it has no norm, or we'd write $||D||=\infty$ (and I promise this is a failure of $D$, not of our definitions). Put another way, $D$** is not bounded as a linear operator**, since it can stretch functions by an arbitrary amount. 

But now let's return to $I$. We said that $||I||=10$ (if we're defining it relative to the $L^\infty$-norm on $X=[-10,10]$), but isn't $(1-x)^{-1}=1+x+x^2+...$ only true when $|x|<1$? For real numbers, yes, but **for operators, something magical happens: **$||I^n|| \neq ||I||^n$! (Its like there's a whole [algebra of these operators](https://en.wikipedia.org/wiki/Operator_algebra)...) 

In fact, you can show that $||I^n||$ assumes its maximum value when applied to the constant function $f(x)=1$, and hence have $||I^n||=\frac{1}{n!}||I||^n$. Since $n!$ grows faster than exponential functions, $||I^n||$ is converging to 0 quickly, so $1+I+I^2+I^3+...$ is a Cauchy sum, and it is then straightforward to show that the limit is the multiplicative inverse of $1-I$. **Thus, **$(1-I)^{-1}=1+I+I^2+...$** is a valid expression that you can apply to any continuous (or bounded) function **$f$** on any compact set **$X$**.** This convergence happens regardless of the choice of the compact set, though it will happen at different rates, analogous to [uniform convergence on compact sets](https://en.wikipedia.org/wiki/Compact_convergence).

Summary 
--------

*   Writing $D$ for derivative and $I$ for integral, we showed that $(1-D)^{-1}=1+D+D^2...$ can fail, even though $(1-I)^{-1}=1+I+I^2...$ is always true.
*   To explain this, we have to show that $I$ is fundamentally better behaved than $D$, in a way analogous to $|x|<1$.
*   We built this up in two steps. First, we defined the $L^\infty$-norm for real-valued functions, which lets you say how "large" those functions are. Then, we extended this to function-valued functions (operators), having to make two slight modifications along the way.
*   With this machinery in place, we could show that $||I||<\infty$, or we can say that $I$ is *bounded*. The resulting norm depends on the domain of the functions $f$ under consideration, but any compact domain is allowable. Also, since $||I^n||=\frac{1}{n!}||I||^n$, the exact value doesn't matter since the norm of each term goes to 0.
*   Since $||I^n|| \rightarrow 0$ sufficiently quickly, we can say that $1+I+I^2+I^3+...$ is Cauchy as a sequence of operators. In other words, if you apply the partial sums $1, 1+I, 1+I+I^2, ...$ as operators to any function $f$, the functions $f, f+If, f+If+I^2f, ...$ will converge with respect to the $L^\infty$-norm. Writing $Lf$ for the function they converge to, it follows that $(1-I)Lf=f$, so we may write $L=1+I+I^2+I^3+...=(1-I)^{-1}$ as a statement about linear operators.
*   In contrast, $D$ is *unbounded* as an operator, meaning $||D||=\infty$. Thus algebra tricks like $(1-D)^{-1}=1+D+D^2...$ will break down if you put in the wrong function $f$.