Part II

Super simple view: In math, there are
sets with elements. That's the logical
foundation of essentially all of current
pure/applied math. The details are in
Zermalo-Fraenkel axiomatic set theory
assuming the axiom of choice. E.g., can
construct from sets a set that looks like
the real numbers we knew about in the 9th
grade. Soon define ordered pairs and,
then, functions. After that, a huge
fraction of everything is functions. The
proofs -- as actually written but without
the crucial intuitive ideas that permitted
finding the proof -- are all essentially
just symbol substitution as in basic logic
and Whitehead and Russell. Warning: Any
math you write for publication should be
easily translated back to just sets and
symbol substitution; that and nothing else
is the criterion. If you also want the
proof to be readable by humans, there is
more. E.g., in the proof might mention
one by one each theorem assumption and
where it gets used.

(iv) Learn linear algebra. Really the
subject grew out of Gauss elimination for
solving systems of linear equations --
with some additional attention to
numerical stability (e.g., partial
pivoting, double precision inner product
accumulation, and iterative improvement)
-- that's nearly always still the way to
do it. It's fun to program a good Gauss
elimination routine, e.g., just in C.

There see clearly that any such system of
equations has none, one, or infinitely
many solutions. Later will discover that
the set of all the solutions is an affine
subspace, that is, a vector space plus
some one vector; that is, a plane that
does not pass through the origin. And
will discover that the left side with the
unknowns is a linear function -- big time
stuff.

So, that start was a small thing for a
huge future.

Next up, we consider n-tuples of real (or
complex, here and always in linear
algebra) numbers. Then we see how to make
the n-tuples a vector space. They are the
most important of the vector spaces.

But should also see the abstract (with no
mention of n-tuples) definition of a
vector space because (a) the n-tuples are
the most important example, (b) even when
working with just n-tuples often need the
more abstract definition (especially for
subspaces, which are often the real
interest, e.g., hyperplanes in curve
fitting and multivariate statistics), and
(c) there are other important vector
spaces that are not just n-tuples (in
signal processing, sets of random
variables, wave functions in quantum
mechanics, solutions to some differential
equations, and much more).

So, learn about linear independence and
bases (essentially coordinate systems).

Learn about inner products, distance,
angles, and orthogonality. See
generalizations of cosines, e.g., the
Schwarz inequality, and the Pythagorean
theorem.

Learn about eigenvalues and eigen vectors
-- those eigen vectors are often the most
important ones in applications, e.g., your
favorite coordinate axes.

Then for the crown jewel, learn about the
polar decomposition and, thus, singular
values, principal components (e.g., data
compression), the core of the normal
equations in statistics, etc.

There is a remark in G. Simmons that the
two pillars of mathematical analysis are
linearity and continuity. The
superposition in physics is essentially
linearity. In applied math, linearity is
the main tool, the key to the land of milk
and honey. Well, a good course in linear
algebra is a good start on linearity. In
particular, those linear equations solved
with Gauss elimination are linear as in
linear transformations in, right, linear
algebra.

Then, sure, that version of linearity
takes one through much of all of applied
math, e.g., Fourier analysis, the fast
Fourier transform, X-ray diffraction,
Banach space, Hilbert space, the classic
Dunford and Schwartz, Linear Operators,
etc.

So, a Banach space is just a vector space
where the scalars are the real or complex
numbers, there is a norm, that is, a
definition of distance, and the space is
complete in that norm. Complete is
what the rational numbers are not but the
real numbers are. Or, complete means
that a sequence that appears to converge,
that is, converges in the weaker sense of
Cauchy, actually does have something to
converge to and does. E.g., in the
rationals, the approximations to more and
more decimal places of pi have nothing to
converge to but in the reals do.

In theorem proving, nice to have
completeness. But, sure, computing knows
next to nothing about completeness because
we compute essentially only with rational
numbers. So, can do a lot of work without
completeness. Indeed, in applications,
often we are just approximating, and the
rationals can get as close as we please to
pi!

Banach spaces are not trivial or useless:
E.g., based on the Hahn-Banach theorem,
there is the grand applied math dessert
buffet

David G. Luenberger, Optimization by
Vector Space Methods, John Wiley and
Sons.

A Hilbert space is a Banach space where
the norm comes from an inner product.

E.g., the set of all real valued random
variables X such that E[X^2] is finite
form a Hilbert space. That the space is
complete for those random variables is
nearly mind blowing; actually the proof is
short.

(v) There remains Baby Rudin, Principles
of Mathematical Analysis.

There see calculus done with essentially
full care, as theorems and proofs. So,
again, get lessons in how to write
theorems and proofs on the way to being a
good mathematician.

The main content of the book is just
showing that a real valued continuous
function defined on a closed interval of
the reals has a Riemann integral.

The key is that that interval is
compact. So, learn about compactness,
which is of quite general usefulness.

Then with compactness and continuity, have
uniform continuity. Now the doors to
grandeur start to open: That the Riemann
integral works is a short proof. And,
later in the book, get the three epsilon
proof that the uniform limit of continuous
functions is a continuous function (was a
question on one of my Ph.D. qualifying
exams -- from baby Rudin, I got it!).

Compactness is so powerful that it is
nearly the same as just finiteness --
there's a famous, old paper on that.

Well for a positive integer n and the set
of real number R, in R^n a set is compact
if and only if it is closed and bounded.
Now we are cleaning up the Riemann
integral and a lot of associated stuff.

At the back, Rudin gives a nice, short
definition of a set of the real numbers
that has measure zero (without really
getting deep into measure theory) and,
then, shows the Riemann integral exists if
and only if the function is continuous
everywhere except on a set of measure
zero. Nice. Now an exercise is to find a
function that is differentiable but whose
derivative is not Riemann integrable.
Might look in Gelbaum and Olmstead,
Counterexamples in Analysis.

Also the later editions of baby Rudin
cover the exterior algebra of differential
forms. That material is of interest in
differential geometry, some applications,
and general relativity.

That's an overview of what baby Rudin is
about.

Go through that book carefully and will
come out with (a) some good knowledge of
the "principles" of the analysis part of
pure math and (b) much better skills at
doing math derivations, definitions,
theorems, and proofs. If you want to
write and publish new proofs for, say, AI,
baby Rudin is one of your best mentors,
maybe your Fairy Godmother?

For being an applied mathematician, sure,
you already guessed, and you were correct,
that essentially always in practice the
Riemann integral exists; so, why sweat the
details? Okay, then just use baby Rudin
to learn about compactness, continuity,
uniform continuity, measure zero, and more
on how to write proofs. And, really,
focusing just on compactness, continuity,
etc., can pull that off in a few, nice
weekends, maybe just one. Then look at
that result near the back on the uniform
limit of continuous functions is
continuous to see how to do such work.

Sure, Rudin discusses compactness, etc. on
metric spaces. Well, easily enough, the
set of real numbers R is also a metric
space! And for positive integer n, so is
R^n.

So, why say metric space instead of, say,
just R^n? Well, first, the theory is
cleaner because a metric space has so
many fewer assumptions than R^n so that
can see more clearly just what assumptions
make the results true. Second, maybe some
fine day all you will have is just a
metric space and, then, can still use the
results -- don't hold your breath while
waiting for a significant application,
either pure or applied, with a non-trivial
metric space that isn't also much more!
Or, proving the stuff in a metric space is
no more difficult, more general, and
maybe, and actually, more useful.

Or, maybe math had the results in R^n and
then invented a metric space just to have
a place to have just enough to make the
results true! So, a metric space was
invented to have the least assumptions
needed for proving the results; so, what
came first were the results, and the
metric space definition came later.
Maybe!