# Inner product/K/Orthogonal basis/Introduction/Section
## Definition

Let $V$ be a ${\mathbb {K} }$-vector space, endowed with an inner product. A basis $v_{i}$, $i\in I$, of $V$ is called an orthogonal basis if
    $\left\langle v_{i},v_{j}\right\rangle =0{\text{ for }}i\neq j$

## Definition

Let $V$ be a ${\mathbb {K} }$-vector space, endowed with an inner product $\left\langle -,-\right\rangle$. A basis $v_{i}$, $i\in I$, of $V$ is called an orthonormal basis if
    $\left\langle v_{i},v_{i}\right\rangle =1{\text{ for all }}i\in I{\text{ and }}\left\langle v_{i},v_{j}\right\rangle =0{\text{ for }}i\neq j$
The elements in an orthonormal basis have norm $1$, and they are orthogonal to each other. Hence, a orthonormal basis is a orthogonal basis, that also satisfies the norm condition
    $\Vert {v_{i}}\Vert ={\sqrt {\left\langle v_{i},v_{i}\right\rangle }}=1\,.$
It is easy to transform an orthogonal basis to an orthonormal basis, by replacing every $v_{i}$ by its normalization ${\frac {v_{i}}{\Vert {v_{i}}\Vert }}$ (as $v_{i}$ is part of a basis, its norm is not $0$). A family of vectors, all of norm $1$, and orthogonal to each other, but not necessarily a basis, is called an orthonormal system.

## Lemma

Let $V$ be a ${\mathbb {K} }$-vector space, endowed with an inner product, and let $u_{i}$, $i\in I$, be an orthonormal basis of $V$. Then the coefficients of a vector $v$, with respect to this basis, are given by
    $v=\sum _{i\in I}\left\langle v,u_{i}\right\rangle u_{i}\,.$

### Proof

Since we have a basis, there exists a unique representation
    $v=\sum _{j\in I}a_{j}u_{j}\,$
(where all $a_{j}$ are $0$ up to finitely many). Therefore, the claim follows from
    $\left\langle v,u_{i}\right\rangle =\left\langle \sum _{j\in I}a_{j}u_{j},u_{i}\right\rangle =\sum _{j\in I}a_{j}\left\langle u_{j},u_{i}\right\rangle =a_{i}\,.$
$\Box$
We will mainly consider orthonormal bases in the finite-dimensional case. In $\mathbb {R} ^{n}$, the standard basis is a orthonormal basis. In the plane $\mathbb {R} ^{2}$, any orthonormal basis is of the form ${\begin{pmatrix}a\\b\end{pmatrix}},{\begin{pmatrix}-b\\a\end{pmatrix}}$ or of the form ${\begin{pmatrix}a\\b\end{pmatrix}},{\begin{pmatrix}b\\-a\end{pmatrix}}$, where $a^{2}+b^{2}=1$ holds. For example, ${\begin{pmatrix}{\frac {3}{5}}\\{\frac {4}{5}}\end{pmatrix}},{\begin{pmatrix}-{\frac {4}{5}}\\{\frac {3}{5}}\end{pmatrix}}$ is an orthonormal basis. The following Gram–Schmidt orthonormalization describes a method to construct, starting with any basis of a finite-dimensional vector space, an orthonormal basis that generates the same flag of linear subspaces.

## Theorem

Let $V$ be a finite-dimensional ${\mathbb {K} }$-vector space, endowed with an inner product, and let $v_{1},v_{2},\ldots ,v_{n}$ be a basis of $V$. Then there exists an orthonormal basis $u_{1},u_{2},\ldots ,u_{n}$ of $V$ with
    $\langle v_{1},v_{2},\ldots ,v_{i}\rangle =\langle u_{1},u_{2},\ldots ,u_{i}\rangle \,$
for alle
$i=1,\ldots ,n$

### Proof

We prove the statement by induction over $i$, that is, we construct successively a family of orthonormal vectors spanning the same linear subspaces. For $i=1$, we just have to normalize $v_{1}$, that is, we replace it by $u_{1}={\frac {v_{1}}{\Vert {v_{1}}\Vert }}$. Now suppose that the statement is already proven for $i$. Let a family of orthonormal vectors $u_{1},\ldots ,u_{i}$ fulfilling $\langle u_{1},\ldots ,u_{i}\rangle =\langle v_{1},\ldots ,v_{i}\rangle$ be already constructed. We set
    $w_{i+1}=v_{i+1}-\left\langle v_{i+1},u_{1}\right\rangle u_{1}-\cdots -\left\langle v_{i+1},u_{i}\right\rangle u_{i}\,.$
Due to
    ${\begin{aligned}\left\langle w_{i+1},u_{j}\right\rangle &=\left\langle v_{i+1}-\left\langle v_{i+1},u_{1}\right\rangle u_{1}-\cdots -\left\langle v_{i+1},u_{i}\right\rangle u_{i},u_{j}\right\rangle \\&=\left\langle v_{i+1},u_{j}\right\rangle -\sum _{k\leq i,k\neq j}\left\langle v_{i+1},u_{k}\right\rangle \left\langle u_{k},u_{j}\right\rangle -\left\langle v_{i+1},u_{j}\right\rangle \left\langle u_{j},u_{j}\right\rangle \\&=\left\langle v_{i+1},u_{j}\right\rangle -\left\langle v_{i+1},u_{j}\right\rangle \\&=0,\end{aligned}}$
this vector is orthgonal to all $u_{1},\ldots ,u_{i}$, and also
    $\langle u_{1},\ldots ,u_{i},w_{i+1}\rangle =\langle v_{1},\ldots ,v_{i},v_{i+1}\rangle \,$
holds. By normalizing $w_{i+1}$, we obtain $u_{i+1}$.
$\Box$

## Example

Let $V$ be the kernel of the linear mapping
    $\mathbb {R} ^{3}\longrightarrow \mathbb {R} ,(x,y,z)\longmapsto 2x+3y-z.$
As a linear subspace of $\mathbb {R} ^{3}$, $V$ carries the induced inner product. We want to determine an orthonormal basis of $V$. For this, we consider the basis consisting of the vectors
    $v_{1}={\begin{pmatrix}1\\0\\2\end{pmatrix}}\,\,{\text{ and }}\,\,v_{2}={\begin{pmatrix}0\\1\\3\end{pmatrix}}.$
We have $\Vert {v_{1}}\Vert ={\sqrt {5}}$; therefore,
    $u_{1}={\frac {v_{1}}{\Vert {v_{1}}\Vert }}={\begin{pmatrix}{\frac {1}{\sqrt {5}}}\\0\\{\frac {2}{\sqrt {5}}}\end{pmatrix}}\,$
is the corresponding normed vector. According to orthonormalization process, we set
    ${\begin{aligned}w_{2}&=v_{2}-\left\langle v_{2},u_{1}\right\rangle u_{1}\\&={\begin{pmatrix}0\\1\\3\end{pmatrix}}-\left\langle {\begin{pmatrix}0\\1\\3\end{pmatrix}},{\begin{pmatrix}{\frac {1}{\sqrt {5}}}\\0\\{\frac {2}{\sqrt {5}}}\end{pmatrix}}\right\rangle {\begin{pmatrix}{\frac {1}{\sqrt {5}}}\\0\\{\frac {2}{\sqrt {5}}}\end{pmatrix}}\\&={\begin{pmatrix}0\\1\\3\end{pmatrix}}-{\frac {6}{\sqrt {5}}}{\begin{pmatrix}{\frac {1}{\sqrt {5}}}\\0\\{\frac {2}{\sqrt {5}}}\end{pmatrix}}\\&={\begin{pmatrix}0\\1\\3\end{pmatrix}}-{\begin{pmatrix}{\frac {6}{5}}\\0\\{\frac {12}{5}}\end{pmatrix}}\\&={\begin{pmatrix}-{\frac {6}{5}}\\1\\{\frac {3}{5}}\end{pmatrix}}.\end{aligned}}$
We have
    $\Vert {w_{2}}\Vert =\Vert {\begin{pmatrix}-{\frac {6}{5}}\\1\\{\frac {3}{5}}\end{pmatrix}}\Vert ={\sqrt {{\frac {36}{25}}+1+{\frac {9}{25}}}}={\sqrt {\frac {70}{25}}}={\frac {\sqrt {14}}{\sqrt {5}}}\,.$
Therefore,
    $u_{2}={\frac {\sqrt {5}}{\sqrt {14}}}{\begin{pmatrix}-{\frac {6}{5}}\\1\\{\frac {3}{5}}\end{pmatrix}}={\begin{pmatrix}-{\frac {6}{\sqrt {70}}}\\{\frac {\sqrt {5}}{\sqrt {14}}}\\{\frac {3}{\sqrt {70}}}\end{pmatrix}}\,$
is the second vector of the orthonormal basis.

## Corollary

Let $V$ be a finite-dimensional ${\mathbb {K} }$-vector space, endowed with an inner product. Then there exists an orthonormal basis
$V$

### Proof

This follows directly from fact.
$\Box$
Therefore, in a finite-dimensional vector space with an inner product, one can always extend a given orthonormal system to a orthonormal basis, see exercise.

## Corollary

Let $V$ be a finite-dimensional ${\mathbb {K} }$-vector space, endowed with an inner product, and let $U\subseteq V$ denote a linear subspace. Then, we have
    $V=U\oplus U^{\perp }\,,$
that is, $V$ is the direct sum of $U$ and its

### Proof

From $u\in U\cap U^{\perp }$ we get directly
    $\left\langle u,u\right\rangle =0\,,$
thus $u=0$. This means that the sum direct. Let $u_{1},\ldots ,u_{k}$ be an orthonormal basis of $U$. We extend it to an orthonormal basis $u_{1},\ldots ,u_{n}$ of $V$. Then we have
    $U^{\perp }=\langle u_{k+1},\ldots ,u_{n}\rangle \,;$
Therefore, $V$ is the sum of the linear subspaces.
$\Box$
For the following statement, compare also fact and exercise.

## Corollary

Let $V$ be a ${\mathbb {K} }$-vector space, endowed with an inner product
$\left\langle -,-\right\rangle$
* For a linear subspace $U\subseteq U'\subseteq V$, we have
    $U^{\perp }\supseteq U'^{\perp }\,.$
* We have $0^{\perp }=V$ and $V^{\perp }=0$.
* Let $V$ be finite-dimensional. Then we have
    ${\left(U^{\perp }\right)}^{\perp }=U\,.$
* Let $V$ be finite-dimensional. Then we have
    $\dim _{K}{\left(U^{\perp }\right)}=\dim _{K}{\left(V\right)}-\dim _{K}{\left(U\right)}\,.$

### Proof

$\Box$
* ↑ Here, $\langle -\rangle$ denotes the linear subspace spanned by the vectors, not the inner product.
* ↑ Often, it is computationally better, first to orthogonalize and to normalize at the very end, see example.