# Linear algebra (Osnabrück 2024-2025)/Part II/Lecture 31/refcontrol

    Vector spaces with an inner product
In $\mathbb {R} ^{n}$, we can add vectors and multiply them with a scalar. Moreover, a vector has a certain length, and the relation between two vectors is expressed by the angle between them. Length and angle can be made precise with the concept of an inner product. In order to introduce this, a real vector space or a complex vector space must be given. We want to discuss both cases in parallel, and we we use the symbol ${\mathbb {K} }$ to denote $\mathbb {R}$ or $\mathbb {C}$. For $z\in \mathbb {C}$, ${\overline {z}}$ means the complex-conjugatedMDLD/complex-conjugated number; for $z\in \mathbb {R}$, this is just the number itself.

## Definition

Let $V$ be a ${\mathbb {K} }$-vector space.MDLD/vector space An inner product on $V$ is a mapping
    $V\times V\longrightarrow {\mathbb {K} },(v,w)\longmapsto \left\langle v,w\right\rangle ,$
satisfying the following properties:
* We have for all $\lambda _{1},\lambda _{2}\in {\mathbb {K} }$, $x_{1},x_{2},y\in V$, and for all $\lambda _{1},\lambda _{2}\in {\mathbb {K} }$, $x,y_{1},y_{2}\in V$.
    $\left\langle \lambda _{1}x_{1}+\lambda _{2}x_{2},y\right\rangle =\lambda _{1}\left\langle x_{1},y\right\rangle +\lambda _{2}\left\langle x_{2},y\right\rangle \,$
    $\left\langle x,\lambda _{1}y_{1}+\lambda _{2}y_{2}\right\rangle ={\overline {\lambda _{1}}}\left\langle x,y_{1}\right\rangle +{\overline {\lambda _{2}}}\left\langle x,y_{2}\right\rangle \,$
* We have for all $v,w\in V$.
    $\left\langle v,w\right\rangle ={\overline {\left\langle w,v\right\rangle }}\,$
* We have $\left\langle v,v\right\rangle \geq 0$ for all $v\in V$, and $\left\langle v,v\right\rangle =0$ if and only if $v=0$ holds.
The used properties are called, in the real case, bilinear (which is just another name for multilinear,MDLD/multilinear when we are dealing with the product of two vector spaces), symmetry and positive-definiteness. In the complex case, we call the properties sesquilinear and hermitian. This looks a bit complicated at first sight but is necessary to ensure that also in the complex case we get positive definiteness and a reasonable concept of distance.

## Example Create referencenumber

On $\mathbb {R} ^{n}$, the mapping
    $\mathbb {R} ^{n}\times \mathbb {R} ^{n}\longrightarrow \mathbb {R} ,(v,w)=({\left(v_{1},\ldots ,v_{n}\right)},{\left(w_{1},\ldots ,w_{n}\right)})\longmapsto \sum _{i=1}^{n}v_{i}w_{i},$
defines an inner product,MDLD/inner product which is called the standard inner product. Simple computations show that this is indeed an inner product.
For example, in $\mathbb {R} ^{3}$, endowed with the standard inner product, we have
    $\left\langle {\begin{pmatrix}3\\-5\\2\end{pmatrix}},{\begin{pmatrix}-1\\4\\6\end{pmatrix}}\right\rangle =3\cdot (-1)-5\cdot 4+2\cdot 6=-11\,.$

## Definition

A real,MDLD/real finite-dimensionalMDLD/finite-dimensional (vs) vector spaceMDLD/vector space that is endowed with an inner product,MDLD/inner product (R)
For a vector space $V$, endowed with an inner product, every linear subspaceMDLD/linear subspace $U\subseteq V$ has again an inner scalar product, by restricting the inner product from $V$. In particular, for an euclidean vector space, every linear subspace $U\subseteq V$ is again an euclidean vector space. Therefore, every linear subspace $U\subseteq \mathbb {R} ^{n}$ carries the restricted standard inner product. Because there is always an isomorphismMDLD/isomorphism (vs) $U\cong \mathbb {R} ^{m}$, we can also transfer the standard inner product from $\mathbb {R} ^{m}$ to $U$. However, the result depends on the isomorphism chosen, and there is, in general, no relationship with the restricted standard inner product.

## Definition

On $\mathbb {C} ^{n}$, we call the inner productMDLD/inner product (C) given by
    $\left\langle u,v\right\rangle :=\sum _{i=1}^{n}u_{i}{\overline {v_{i}}}\,$
the
For example, we have
    ${\begin{aligned}\left\langle {\begin{pmatrix}4-3{\mathrm {i} }\\2+7{\mathrm {i} }\end{pmatrix}},{\begin{pmatrix}-2+5{\mathrm {i} }\\3-6{\mathrm {i} }\end{pmatrix}}\right\rangle &=(4-3{\mathrm {i} })\cdot {\overline {-2+5{\mathrm {i} }}}+(2+7{\mathrm {i} })\cdot {\overline {3-6{\mathrm {i} }}}\\&=(4-3{\mathrm {i} })\cdot (-2-5{\mathrm {i} })+(2+7{\mathrm {i} })\cdot (3+6{\mathrm {i} })\\&=-23-14{\mathrm {i} }-36+33{\mathrm {i} }\\&=-59+19{\mathrm {i} }.\end{aligned}}$

## Remark

If we consider a complex vector space $V$, endowed with an inner productMDLD/inner product (C) $\left\langle -,-\right\rangle$, as a real vector space, then the real partMDLD/real part
    $\operatorname {Re} \,{\left(\left\langle v,w\right\rangle \right)}$
is a real inner product, see exercise *****. Because of
    ${\begin{aligned}\left\langle v,w\right\rangle &=\operatorname {Re} \,{\left(\left\langle v,w\right\rangle \right)}+{\mathrm {i} }\operatorname {Im} \,{\left(\left\langle v,w\right\rangle \right)}\\&=\operatorname {Re} \,{\left(\left\langle v,w\right\rangle \right)}-{\mathrm {i} }\operatorname {Re} \,{\left({\mathrm {i} }\left\langle v,w\right\rangle \right)}\\&=\operatorname {Re} \,{\left(\left\langle v,w\right\rangle \right)}-{\mathrm {i} }\operatorname {Re} \,{\left(\left\langle iv,w\right\rangle \right)},\end{aligned}}$
we can reconstruct from the real part the original inner product.

## Example Create referencenumber

Let $[a,b]$ be a closed real interval with $a<b$, and let
    $V={\left\{f:[a,b]\rightarrow \mathbb {C} \mid f{\text{ continuous}}\right\}}\,,$
endowed with pointwise addition and scalar multiplication. Setting
    $\left\langle f,g\right\rangle :=\int _{a}^{b}f(t){\overline {g(t)}}dt\,,$
we obtain an inner productMDLD/inner product on $V$. Additivity follows from
    $\left\langle f_{1}+f_{2},g\right\rangle =\int _{a}^{b}(f_{1}(t)+f_{2}(t)){\overline {g(t)}}dt=\int _{a}^{b}f_{1}(t){\overline {g(t)}}dt+\int _{a}^{b}f_{2}(t){\overline {g(t)}}dt=\left\langle f_{1},g\right\rangle +\left\langle f_{2},g\right\rangle \,.$
It is also positive definite: If $f$ is not the zero function, then let $s\in [a,b]$ denote a point with $f(s)\neq 0$. Therefore, $\vert {f(s)}\vert >0$, and due to the continuity of $f$ there exists a neighborhood $[s-\epsilon ,s+\epsilon ]\cap [a,b]$ of length $\geq \epsilon$, where
    $\vert {f(t)}\vert \geq c>0\,$
holds for some $c\in \mathbb {R} _{+}$ (one can take $c={\frac {\vert {f(s)}\vert }{2}}$), Hence,
    $\left\langle f,f\right\rangle =\int _{a}^{b}\vert {f(t)}\vert ^{2}dt\geq \int _{\max {\left(a,s-\epsilon \right)}}^{\min {\left(b,s+\epsilon \right)}}\vert {f(t)}\vert ^{2}dt\geq \epsilon c^{2}>0\,$
is positive.
    Norm
If an inner product is given, then we can define the length of a vector, and then also the distance between two vectors.

## Definition

Let $V$ denote a vector spaceMDLD/vector space over ${\mathbb {K} }$, endowed with an inner productMDLD/inner product $\left\langle -,-\right\rangle$. For a vector $v\in V$, we call the real number
    $\Vert {v}\Vert ={\sqrt {\left\langle v,v\right\rangle }}\,$
$v$
The inner product $\left\langle v,v\right\rangle$ is always real and not negative; therefore, its square root is a uniquely determined real number. For a complex vector space with an inner product, it does not make any difference whether we determine the norm directly, or via the underlying real vector space, see exercise *****.

## TheoremTheorem 31.8 change

Let $V$ denote a vector spaceMDLD/vector space over ${\mathbb {K} }$, endowed with an inner productMDLD/inner product $\left\langle -,-\right\rangle$. Let $\Vert {-}\Vert$ denote the associated norm.MDLD/norm (inner product) Then the Cauchy-Schwarz estimate holds, that is,
    $\vert {\left\langle v,w\right\rangle }\vert \leq \Vert {v}\Vert \cdot \Vert {w}\Vert \,$
for all
$v,w\in V$

### Proof

For $w=0$, the statement holds. So suppose $w\neq 0$, hence, also $\Vert {w}\Vert \neq 0$ holds. Therefore, we have the estimates
    ${\begin{aligned}0&\leq \left\langle v-{\frac {\left\langle v,w\right\rangle }{\Vert {w}\Vert ^{2}}}w,v-{\frac {\left\langle v,w\right\rangle }{\Vert {w}\Vert ^{2}}}w\right\rangle \\&=\left\langle v,v\right\rangle -{\frac {\left\langle v,w\right\rangle }{\Vert {w}\Vert ^{2}}}\left\langle w,v\right\rangle -{\frac {\overline {\left\langle v,w\right\rangle }}{\Vert {w}\Vert ^{2}}}\left\langle v,w\right\rangle +{\frac {\left\langle v,w\right\rangle {\overline {\left\langle v,w\right\rangle }}}{\Vert {w}\Vert ^{4}}}\left\langle w,w\right\rangle \\&=\left\langle v,v\right\rangle -{\frac {\left\langle v,w\right\rangle }{\Vert {w}\Vert ^{2}}}{\overline {\left\langle v,w\right\rangle }}-{\frac {\overline {\left\langle v,w\right\rangle }}{\Vert {w}\Vert ^{2}}}\left\langle v,w\right\rangle +{\frac {\left\langle v,w\right\rangle {\overline {\left\langle v,w\right\rangle }}}{\Vert {w}\Vert ^{2}}}\\&=\left\langle v,v\right\rangle -{\frac {\left\langle v,w\right\rangle {\overline {\left\langle v,w\right\rangle }}}{\Vert {w}\Vert ^{2}}}\\&=\left\langle v,v\right\rangle -{\frac {\vert {\left\langle v,w\right\rangle }\vert ^{2}}{\Vert {w}\Vert ^{2}}}.\end{aligned}}$
Multiplication with $\Vert {w}\Vert ^{2}$ and taking the square root yields the result.
$\Box$

## LemmaLemma 31.9 change

Let $V$ denote a vector spaceMDLD/vector space over ${\mathbb {K} }$, endowed with an inner productMDLD/inner product $\left\langle -,-\right\rangle$. Then the corresponding normMDLD/norm (inner product)
* We have $\Vert {v}\Vert \geq 0$.
* We have $\Vert {v}\Vert =0$ if and only if $v=0$.
* For $\lambda \in {\mathbb {K} }$ and $v\in V$, we have
    $\Vert {\lambda v}\Vert =\vert {\lambda }\vert \cdot \Vert {v}\Vert \,.$
* For $v,w\in V$, we have
    $\Vert {v+w}\Vert \leq \Vert {v}\Vert +\Vert {w}\Vert \,.$

### Proof

The first two properties follow directly from the definition of an inner product.MDLD/inner product (K) The compatibility with multiplication follows from
    $\Vert {\lambda v}\Vert ^{2}=\left\langle \lambda v,\lambda v\right\rangle =\lambda \left\langle v,\lambda v\right\rangle =\lambda {\overline {\lambda }}\left\langle v,v\right\rangle =\vert {\lambda }\vert ^{2}\Vert {v}\Vert ^{2}\,.$
In order to prove the triangle estimate, we write
    ${\begin{aligned}\Vert {v+w}\Vert ^{2}&=\left\langle v+w,v+w\right\rangle \\&=\Vert {v}\Vert ^{2}+\Vert {w}\Vert ^{2}+\left\langle v,w\right\rangle +{\overline {\left\langle v,w\right\rangle }}\\&=\Vert {v}\Vert ^{2}+\Vert {w}\Vert ^{2}+2\operatorname {Re} \,{\left(\left\langle v,w\right\rangle \right)}\\&\leq \Vert {v}\Vert ^{2}+\Vert {w}\Vert ^{2}+2\vert {\left\langle v,w\right\rangle }\vert .\end{aligned}}$
Due to Theorem 31.8 , this is $\leq {\left(\Vert {v}\Vert +\Vert {w}\Vert \right)}^{2}$. This estimate transfers to the square roots.
$\Box$
With the following statement, the polarization identity, we can reconstruct the inner product from its associated norm.

## LemmaLemma 31.10 change

Let $V$ denote a vector spaceMDLD/vector space over ${\mathbb {K} }$, endowed with an inner productMDLD/inner product $\left\langle -,-\right\rangle$. Let $\Vert {-}\Vert$ denote the associated norm.MDLD/norm (inner product) Then, in case ${\mathbb {K} }=\mathbb {R}$, the relation
    $\left\langle v,w\right\rangle ={\frac {1}{2}}{\left(\Vert {v+w}\Vert ^{2}-\Vert {v}\Vert ^{2}-\Vert {w}\Vert ^{2}\right)}\,$
holds, and, in case ${\mathbb {K} }=\mathbb {C}$, the relation
    $\left\langle v,w\right\rangle ={\frac {1}{4}}{\left(\Vert {v+w}\Vert ^{2}-\Vert {v-w}\Vert ^{2}+{\mathrm {i} }\Vert {v+{\mathrm {i} }w}\Vert ^{2}-{\mathrm {i} }\Vert {v-{\mathrm {i} }w}\Vert ^{2}\right)}\,$

### Proof

$\Box$
    Normed vector spaces
Due to Lemma 31.9 , the norm associated to an inner product is a norm in the sense of the following definition. In particular, a vector space with an inner product is a normed vector space.

## Definition

Let $V$ be a ${\mathbb {K} }$-vector space.MDLD/vector space A mappingMDLD/mapping
    $\Vert {-}\Vert \colon V\longrightarrow \mathbb {R} ,v\longmapsto \Vert {v}\Vert ,$
is called norm, if the following properties hold.
* We have $\Vert {v}\Vert \geq 0$ for all $v\in V$.
* We have $\Vert {v}\Vert =0$ if and only if $v=0$.
* For $\lambda \in {\mathbb {K} }$ and $v\in V$, we have
    $\Vert {\lambda v}\Vert =\vert {\lambda }\vert \cdot \Vert {v}\Vert \,.$
* For $v,w\in V$, we have
    $\Vert {v+w}\Vert \leq \Vert {v}\Vert +\Vert {w}\Vert \,.$

## Definition

A ${\mathbb {K} }$-vector spaceMDLD/vector space is called a normed vector space if a normMDLD/norm
$\Vert {-}\Vert$
On a euclidean vector space, the norm given via the the inner product is also called the euclidean norm. For $V=\mathbb {R} ^{n}$, endowed with the standard inner product, we have
    $\Vert {v}\Vert ={\sqrt {\sum _{i=1}^{n}v_{i}^{2}}}\,.$

## Example Create referencenumber

In ${\mathbb {K} }^{n}$, taking
    $\Vert {x}\Vert _{\rm {max}}:={\max {\left(\vert {x_{i}}\vert ,i=1,\ldots ,n\right)}}\,,$
a normMDLD/norm is defined, which is called the maximum norm.

## Example Create referencenumber

In ${\mathbb {K} }^{n}$, taking
    $\mid \mid \!x\!\mid \mid _{\operatorname {sum} }:=\sum _{i=1}^{n}\vert {x_{i}}\vert \,,$
defines a norm,MDLD/norm which is called the sum norm.
For a vector $v\in V$, $v\neq 0$, in a normed vector space $V$, the vector ${\frac {v}{\Vert {v}\Vert }}$ is called the corresponding normalized. Such a normalized vector has norm $1$. Passing to the normalized vector is also called normalization.
    Normed spaces as metric spaces

## Definition

Let $M$ be a set. A mapping $d\colon M\times M\rightarrow \mathbb {R}$ is called a metric (or a distance function), if for all $x,y,z\in M$ the following conditions hold:
* $d{\left(x,y\right)}=0$ if and only if $x=y$ (positivity),
* $d{\left(x,y\right)}=d{\left(y,x\right)}$ (symmetry), and
* $d{\left(x,y\right)}\leq d{\left(x,z\right)}+d{\left(z,y\right)}$ (triangle inequality).
A metric space is a pair $(M,d)$, where $M$ is a set and $d\colon M\times M\rightarrow \mathbb {R}$

## Definition

On a normed vector spaceMDLD/normed vector space $V$ with norm $\Vert {-}\Vert$, we define the corresponding metric by
    $d(v,w):=\Vert {v-w}\Vert \,.$
This is indeed a metric.

## LemmaCreate referencenumber

A normed vector spaceMDLD/normed vector space $V$ is with the corresponding metricMDLD/corresponding metric (norm) a

### Proof

$\Box$
In particular, a Euclidean space is a metric space.

## Remark

An affine spaceMDLD/affine space over a normed vector spaceMDLD/normed vector space $V$ is a metric space,MDLD/metric space be setting
    $d(P,Q)=\Vert {\overrightarrow {PQ}}\Vert \,.$
This metric is invariant under translations.MDLD/translations (affine)

## Example Create referencenumber

Let $(M,d)$ be a metric space,MDLD/metric space and let $T\subseteq M$ denote a subset. Then $T$ is also a metric space by setting
    $d_{T}(x,y):=d(x,y)\,$
for all $x,y\in T$. This metric is called the induced metric.
Hence, every subset of an affine space over an Euclidean or normed vector space is a metric space.