<figure><img src="image_1.png" /></figure>

HAL
open science

# Fifteen Methods to Calculate the Inverse of a Matrix

David Pigeon

► To cite this version:

David Pigeon. Fifteen Methods to Calculate the Inverse of a Matrix. 2025. hal-04682079v2

HAL Id: hal-04682079

https://hal.science/hal-04682079v2

Preprint submitted on 30 Jan 2025

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Fifteen Methods to Calculate the Inverse of a Matrix

David Pigeon*

30 janvier 2025

**Keywords** : matrices, inversion, elementary operations, Gauss pivot, *LU* decomposition, cofactor matrix, cofactors, determinant, Cayley-Hamilton, characteristic polynomial, *QR* decomposition, Cholesky decomposition, Schur decomposition, Householder matrices, polar decomposition, Givens rotations.

## Résumé

We present fifteen simple methods to calculate the inverse of a matrix using an example : by the cofactor matrix formula, by the *PLU* and *LU* decompositions, by the Gauss pivot, by performing row operations, by performing column operations, by using the Cayley-Hamilton theorem and a formula derived from this theorem, by using diagonalization, and by using the Schur, Cholesky, *QR*, and polar decompositions.

## Table des matières

<table><tr><td>1</td><td>Cofactor Matrix</td><td>7</td></tr><tr><td>1.1</td><td>Introduction</td><td>7</td></tr><tr><td>1.2</td><td>Example</td><td>9</td></tr><tr><td>2</td><td>PLU and LU Decompositions</td><td>10</td></tr><tr><td>2.1</td><td>Introduction</td><td>10</td></tr><tr><td>2.2</td><td>Example</td><td>15</td></tr><tr><td>2.3</td><td>Example of Decomposition into Elementary Matrices</td><td>16</td></tr><tr><td>3</td><td>Gaussian Elimination</td><td>18</td></tr><tr><td>3.1</td><td>Introduction</td><td>18</td></tr><tr><td>3.2</td><td>Example</td><td>19</td></tr><tr><td>4</td><td>Operation on Rows</td><td>21</td></tr><tr><td>4.1</td><td>Introduction</td><td>21</td></tr><tr><td>4.2</td><td>Example</td><td>21</td></tr><tr><td>4.3</td><td>Product of the Matrices Associated with Elementary Row Operations</td><td>22</td></tr><tr><td>5</td><td>Operations on Columns</td><td>24</td></tr><tr><td>5.1</td><td>Introduction</td><td>24</td></tr><tr><td>5.2</td><td>Example</td><td>25</td></tr><tr><td>5.3</td><td>Product of Matrices Associated with Elementary Operations</td><td>26</td></tr><tr><td>6</td><td>Cayley-Hamilton Theorem</td><td>28</td></tr><tr><td>6.1</td><td>Introduction</td><td>28</td></tr><tr><td>6.2</td><td>Example</td><td>30</td></tr></table>

<footnote>*https://sites.google.com/view/davidpigeon/</footnote>
<table><tbody><tr><td>7</td><td>Formula Derived from the Cayley-Hamilton Theorem</td><td>32</td></tr><tr><td>7.1</td><td>Introduction</td><td>32</td></tr><tr><td>7.2</td><td>Example</td><td>34</td></tr><tr><td>8</td><td>Diagonalization and Minimal Polynomial</td><td>35</td></tr><tr><td>8.1</td><td>Introduction</td><td>35</td></tr><tr><td>8.2</td><td>Example</td><td>39</td></tr><tr><td>9</td><td>Diagonalization and transition matrix</td><td>42</td></tr><tr><td>9.1</td><td>Introduction</td><td>42</td></tr><tr><td>9.2</td><td>Example</td><td>42</td></tr><tr><td>10</td><td>Diagonalization and Orthogonal Matrix</td><td>45</td></tr><tr><td>10.1</td><td>Introduction</td><td>45</td></tr><tr><td>10.2</td><td>Example</td><td>48</td></tr><tr><td>11</td><td>Schur Decomposition</td><td>52</td></tr><tr><td>11.1</td><td>Introduction</td><td>52</td></tr><tr><td>11.2</td><td>Example</td><td>57</td></tr><tr><td>12</td><td>Cholesky Decomposition</td><td>60</td></tr><tr><td>12.1</td><td>Introduction</td><td>60</td></tr><tr><td>12.2</td><td>Example</td><td>63</td></tr><tr><td>13</td><td>QR Decomposition and Gram-Schmidt Orthonormalization</td><td>65</td></tr><tr><td>13.1</td><td>Introduction</td><td>65</td></tr><tr><td>13.2</td><td>Example</td><td>69</td></tr><tr><td>14</td><td>Decomposition RQ and Householder Matrices</td><td>71</td></tr><tr><td>14.1</td><td>Introduction</td><td>71</td></tr><tr><td>14.2</td><td>Example</td><td>74</td></tr><tr><td>15</td><td>Polar Decomposition</td><td>76</td></tr><tr><td>15.1</td><td>Introduction</td><td>76</td></tr><tr><td>15.2</td><td>Example</td><td>79</td></tr><tr><td>A</td><td>Operation Matrices</td><td>81</td></tr><tr><td>A.1</td><td>Symmetric Groups</td><td>81</td></tr><tr><td>A.2</td><td>Determinant of a Matrix</td><td>85</td></tr><tr><td>A.3</td><td>Permutation Matrices</td><td>87</td></tr><tr><td>A.4</td><td>Elementary Operation Matrices</td><td>89</td></tr><tr><td>A.5</td><td>Properties</td><td>90</td></tr><tr><td>A.6</td><td>Case of Triangular Matrices</td><td>93</td></tr><tr><td>B</td><td>Elementary Row Operations</td><td>97</td></tr><tr><td>B.1</td><td>The Three Elementary Operations</td><td>97</td></tr><tr><td>B.2</td><td>A Simple Example</td><td>98</td></tr><tr><td>C</td><td>Elementary Operations on Columns</td><td>99</td></tr><tr><td>C.1</td><td>The Three Elementary Operations</td><td>99</td></tr><tr><td>C.2</td><td>A Simple Example</td><td>100</td></tr></tbody></table>
<table><tr><td>D Givens Rotations</td><td>101</td></tr><tr><td>D.1 Introduction</td><td>101</td></tr><tr><td>D.2 Example</td><td>104</td></tr><tr><td>Références</td><td>107</td></tr></table>
# Introduction

Throughout the text, unless otherwise specified, $n$ and $p$ are non-zero natural numbers, and $\mathbb{R}$ represents the field of real numbers. We denote:

* $\mathcal{M}_{n,p}(\mathbb{R})$ as the set of matrices of size $(n,p)$;

* $\mathcal{M}_n(\mathbb{R}) := \mathcal{M}_{n,n}(\mathbb{R})$ as the set of square matrices of size $n$.

## Definition 0.0.1

Let $M \in \mathcal{M}_n(\mathbb{R})$. We say that $M$ is **invertible** if there exists $N \in \mathcal{M}_n(\mathbb{R})$ such that:

$$MN = I_n \land NM = I_n.$$

We denote $\text{GL}_n(\mathbb{R})$ as the **general linear group of invertible matrices of size $n$**.

If $M$ is invertible, then the matrix $N$ is unique, and we denote it as:

$$M^{-1}.$$

It can be shown that only one of the two equalities is necessary, as we have the following theorem.

## Theorem 0.0.2

Let $M, N \in \mathcal{M}_n(\mathbb{R})$. Suppose that $MN = I_n$. Then we have $NM = I_n$.

Each section of the text consists of two subsections. The first is a theoretical subsection where we study the case in dimension $n$ (for some results, we will prove the theorem(s) in the case $n := 3$), and the second subsection applies these results to the matrix $A$ shown below.

# The Matrix $A$ of the Text

The matrix $A$ to which we apply the theoretical results for the calculation of $A^{-1}$ is the matrix:

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

We will calculate the inverse of $A$ in fifteen different ways<fnref n="1" />:

1. using the cofactor matrix;

2. using the PLU and LU decompositions;

3. applying the Gauss pivot method;

4. performing elementary row operations;

5. performing elementary column operations;

6. using the Cayley-Hamilton theorem;

7. using a formula derived from the Cayley-Hamilton theorem;

<footnote n="1">1. We begin with the methods using the cofactor matrix and the PLU and LU decompositions because they are the basis for several proofs that will follow.</footnote>
8. diagonalizing and using the minimal polynomial;

9. diagonalizing and using the change of basis matrix;

10. diagonalizing and using an orthogonal change of basis matrix;

11. using the Schur decomposition;

12. using the Cholesky decomposition;

13. using the QR decomposition with the Gram-Schmidt method;

14. using the RQ decomposition with Householder matrices;

15. using the polar decomposition.

In all cases, we will obtain that the inverse of A is :

$$A^{-1} = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}$$

The methods used in the text are not necessarily common methods for calculating the inverse of a matrix. The two objectives of the text are :

(1) to practice powerful and commonly used linear algebra tools on a simple example;

(2) to introduce new linear algebra techniques. Some methods will involve extensive calculations. They may seem unnecessary due to their length but can be useful depending on the information available before calculating the inverse.

The end of the text contains four appendices :

A. on useful concepts in linear algebra : the determinant and elementary matrices;

B. on elementary row operations;

C. on elementary column operations;

D. a final technique for calculating the QR decomposition using Givens rotations.

Further information related to this text can be found in the books by Denis Serre [2], Roger A. Horn and Charles R. Johnson [1], and on the Wikipedia pages [3].

## Invertibility of A

We conclude this introduction with the invertibility of the matrix A. We demonstrate in two different ways that A is invertible (see the appendix and the text for the notations used).<fnref n="2" />

(1) By exploiting the 0 in the last row and since the rank is invariant under row and column operations (see sections B and C for the notations used), we have directly :

$$ \begin{align*} \mathrm{rg}(A) &= \mathrm{rg} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \\ &= \mathrm{rg} \begin{pmatrix} 2 & -2 & 1 \\ -2 & 1 & 0 \\ -1 & 2 & 0 \end{pmatrix} \quad L_2 \rightarrow L_2 - 2L_1 \end{align*} $$

<footnote n="2">2. Each of the methods discussed in this text can also be used to demonstrate invertibility : a possible or completed calculation is a criterion for invertibility.</footnote>
$$ 
\begin{aligned}
&= 1 + \operatorname{rg} \begin{pmatrix} -2 & 1 \\ -1 & 2 \end{pmatrix} \\
&= 3
\end{aligned}
$$

because the two rows of the 2 × 2 matrix are non-collinear (thus forming a linearly independent set).

(2) By calculating the determinant with respect to the last column, we have :

$$ 
\begin{aligned}
\det(A) &= \begin{vmatrix} 2 & -3 \\ -1 & 2 \end{vmatrix} - 2 \begin{vmatrix} 2 & -2 \\ -1 & 2 \end{vmatrix} \\
&= 1 - 4 \\
&= -3 \\
&\neq 0
\end{aligned}
$$
## 1 Cofactor Matrix

In this section, we use the standard formula that relates the inverse of a matrix to the transpose of the cofactor matrix, also known as the adjugate (see Appendix A for the necessary information on the symmetric group and the determinant).

### 1.1 Introduction

Let us recall the definition of the cofactor matrix of a square matrix.

**Definition 1.1.1**

Let $M \in \mathcal{M}_n(\mathbb{R})$.

(i) Let $i, j \in \{1, \dots, n\}$.

(a) The minor $\Delta_{i,j}(M)$ of index $(i, j)$ of $M$ is defined as the determinant of the matrix $M$ with row $i$ and column $j$ removed.

(b) The cofactor of index $(i, j)$ of $M$ is defined by :

$$(-1)^{i+j} \Delta_{i,j}(M).$$

(ii) The cofactor matrix of $M$ is the matrix of cofactors defined by :

$$\mathrm{Com}(M) := ((-1)^{i+j} \Delta_{i,j}(M))_{i,j} \in \mathcal{M}_n(\mathbb{R}).$$

We then deduce a method for calculating the inverse using the following standard result.

**Proposition 1.1.2**

Let $M \in \mathcal{M}_n(\mathbb{R})$. We have :

$$\det(M)I_n = M\mathrm{Com}(M)^T = \mathrm{Com}(M)^T M.$$

**Proof.** Let $M := (m_{i,j})_{ij}$. We will show one of the two equalities for $n := 3$. For all $i, j \in \{1, 2, 3\}$, we have :

$$ \begin{aligned} [M\mathrm{Com}(M)^T]_{ij} &= \begin{bmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{bmatrix} \begin{pmatrix} \Delta_{11}(M) & -\Delta_{21}(M) & \Delta_{31}(M) \\ -\Delta_{12}(M) & \Delta_{22}(M) & -\Delta_{32}(M) \\ \Delta_{13}(M) & -\Delta_{23}(M) & \Delta_{33}(M) \end{pmatrix} \end{bmatrix}_{ij} \\ &= \sum_{k=1}^{3} m_{ik} [\mathrm{Com}(M)^T]_{kj} \\ &= \sum_{k=1}^{3} m_{ik} [\mathrm{Com}(M)]_{jk} \\ &= \sum_{k=1}^{3} (-1)^{j+k} m_{ik} \Delta_{j,k}(M) \end{aligned} $$

There are two cases.

(1) Case $i = j$. Let us take, for example, $i := j := 1$. We have :

$$\mathfrak{S}_3 = \{\mathrm{Id}, (12), (13), (23), (123) = (12)(23), (132) = (13)(32)\}$$
$$= : \{\sigma_1, \sigma_2, \sigma_3, \sigma_4, \sigma_5, \sigma_6\}$$

and the signatures of the six permutations are :

$$\varepsilon(\sigma_1) = 1$$

$$\varepsilon(\sigma_2) = -1$$

$$\varepsilon(\sigma_3) = -1$$

$$\varepsilon(\sigma_4) = -1$$

$$\varepsilon(\sigma_5) = 1$$

$$\varepsilon(\sigma_6) = 1$$

We have :

$$
\begin{align*}
& [M \operatorname{Com}(M)^T]_{11} \\
&= \sum_{k=1}^{3} m_{1k} (-1)^{1+k} \Delta_{1,k}(M) \\
&= m_{11} \Delta_{1,1}(M) - m_{12} \Delta_{1,2}(M) + m_{13} \Delta_{1,3}(M) \\
&= m_{11} \begin{vmatrix} m_{22} & m_{23} \\ m_{32} & m_{33} \end{vmatrix} - m_{12} \begin{vmatrix} m_{21} & m_{23} \\ m_{31} & m_{33} \end{vmatrix} + m_{13} \begin{vmatrix} m_{21} & m_{22} \\ m_{31} & m_{32} \end{vmatrix} \\
&= m_{11} m_{22} m_{33} - m_{11} m_{23} m_{32} - m_{12} m_{21} m_{33} + m_{12} m_{23} m_{31} + m_{13} m_{21} m_{32} - m_{13} m_{22} m_{31} \\
&= \varepsilon(\sigma_1) \prod_{i=1}^{3} m_{i,\sigma_1(i)} + \varepsilon(\sigma_4) \prod_{i=1}^{3} m_{i,\sigma_4(i)} + \varepsilon(\sigma_2) \prod_{i=1}^{3} m_{i,\sigma_2(i)} \\
&\quad + \varepsilon(\sigma_5) \prod_{i=1}^{3} m_{i,\sigma_5(i)} + \varepsilon(\sigma_6) \prod_{i=1}^{3} m_{i,\sigma_6(i)} + \varepsilon(\sigma_3) \prod_{i=1}^{3} m_{i,\sigma_3(i)} \\
&= \sum_{k=1}^{6} \varepsilon(\sigma_k) \prod_{i=1}^{3} m_{i,\sigma_k(i)} \\
&= \sum_{\sigma \in \mathfrak{S}_3} \varepsilon(\sigma) \prod_{i=1}^{n} m_{i,\sigma(i)} \\
&= \det(M)
\end{align*}
$$

(2) Case $i \neq j$. Let us take, for example, $i := 1$ and $j := 2$. We have :

$$
\begin{align*}
& [M \operatorname{Com}(M)^T]_{12} \\
&= \sum_{k=1}^{3} m_{1k} (-1)^{2+k} \Delta_{2,k}(M) \\
&= -m_{11} \Delta_{2,1}(M) + m_{12} \Delta_{2,2}(M) - m_{13} \Delta_{2,3}(M) \\
&= -m_{11} \begin{vmatrix} m_{12} & m_{13} \\ m_{32} & m_{33} \end{vmatrix} + m_{12} \begin{vmatrix} m_{11} & m_{13} \\ m_{31} & m_{33} \end{vmatrix} - m_{13} \begin{vmatrix} m_{11} & m_{12} \\ m_{31} & m_{32} \end{vmatrix} \\
&= -m_{11} m_{12} m_{33} + m_{11} m_{32} m_{13} + m_{12} m_{11} m_{33} - m_{12} m_{31} m_{13} - m_{13} m_{11} m_{32} + m_{13} m_{31} m_{12} \\
&= 0
\end{align*}
$$

<figure><img src="image_3.png" /></figure>

We then deduce a formula for calculating the inverse of a matrix.

### Corollary 1.1.3

Let $A \in \operatorname{GL}_n(\mathbb{R})$. We have :

$$A^{-1} = \frac{1}{\det(A)} \operatorname{Com}(A)^T.$$
## 1.2 Example

Let's calculate the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

using the formula from the corollary :

$$A^{-1} = \frac{1}{\det(A)} \mathrm{Com}(A)^T.$$

* Cofactor matrix of A. We have :

$$\begin{align*}
\mathrm{Com}(A) &= \begin{pmatrix}
\Delta_{11}(A) & -\Delta_{12}(A) & \Delta_{13}(A) \\
-\Delta_{21}(A) & \Delta_{22}(A) & -\Delta_{23}(A) \\
\Delta_{31}(A) & -\Delta_{32}(A) & \Delta_{33}(A)
\end{pmatrix} \\
&= \begin{pmatrix}
\begin{vmatrix} -3 & 2 \\ 2 & 0 \end{vmatrix} & - \begin{vmatrix} 2 & 2 \\ -1 & 0 \end{vmatrix} & \begin{vmatrix} 2 & -3 \\ -1 & 2 \end{vmatrix} \\
- \begin{vmatrix} -2 & 1 \\ 2 & 0 \end{vmatrix} & \begin{vmatrix} 2 & 1 \\ -1 & 0 \end{vmatrix} & - \begin{vmatrix} 2 & -2 \\ -1 & 2 \end{vmatrix} \\
\begin{vmatrix} -2 & 1 \\ -3 & 2 \end{vmatrix} & - \begin{vmatrix} 2 & 1 \\ 2 & 2 \end{vmatrix} & \begin{vmatrix} 2 & -2 \\ 2 & -3 \end{vmatrix}
\end{pmatrix} \\
&= \begin{pmatrix}
-4 & -2 & 1 \\
2 & 1 & -2 \\
-1 & -2 & -2
\end{pmatrix}
\end{align*}$$

* Determinant of A. We have :

$$\begin{align*}
\det(A) &= \begin{vmatrix}
2 & -2 & 1 \\
2 & -3 & 2 \\
-1 & 2 & 0
\end{vmatrix} \\
&= 2 \begin{vmatrix}
-3 & 2 \\
2 & 0
\end{vmatrix} - 2 \begin{vmatrix}
-2 & 1 \\
2 & 0
\end{vmatrix} - \begin{vmatrix}
-2 & 1 \\
-3 & 2
\end{vmatrix} \\
&= -8 + 4 + 1 \\
&= -3
\end{align*}$$

Thus, we have :

$$\begin{align*}
A^{-1} &= \frac{1}{\det(A)} \mathrm{Com}(A)^T \\
&= \frac{1}{-3} \begin{pmatrix} -4 & -2 & 1 \\ 2 & 1 & -2 \\ -1 & -2 & -2 \end{pmatrix}^T \\
&= -\frac{1}{3} \begin{pmatrix} -4 & 2 & -1 \\ -2 & 1 & -2 \\ 1 & -2 & -2 \end{pmatrix} \\
&= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}
\end{align*}$$
# 2 PLU and LU Decompositions

In this section, we use PLU and LU decompositions to calculate the inverse of a matrix (see Appendix A for the necessary information on determinants and operation matrices). These decompositions allow us to express an invertible matrix as the product of two triangular matrices, up to multiplication by a permutation matrix. This makes the calculation of the matrix inverse straightforward.

## 2.1 Introduction

We begin with a definition that will be useful for the LU decomposition, namely, the principal minor.

### Definition 2.1.1

Let $M := (m_{ij})_{ij} \in \mathcal{M}_n(\mathbb{R})$. The **principal minors of M** are the determinants of the submatrices :

$$ (m_{ij})_{1\le i,j\le k} $$

for $k \in \{1, \dots, n\}$.

For every $k \in \{1, \dots, n\}$, we denote :

$$ \mu_k(M) := \det((m_{ij})_{1\le i,j\le k}) $$

which is called the **principal minor of M of size k**.

For example, for a matrix :

$$ M := \begin{pmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{pmatrix} $$

the three principal minors are :

$$ \mu_1(M) = \det(m_{11}) = m_{11} $$

$$ \mu_2(M) = \Delta_{33}(M) = \begin{vmatrix} m_{11} & m_{12} \\ m_{21} & m_{22} \end{vmatrix} = m_{11}m_{22} - m_{12}m_{21} $$

$$ \mu_3(M) = \det(M) = \begin{vmatrix} m_{11} & m_{12} & m_{13} \\ m_{21} & m_{22} & m_{23} \\ m_{31} & m_{32} & m_{33} \end{vmatrix} $$

We then have the following result on the existence and uniqueness of PLU and LU decompositions.

### Proposition 2.1.2

(i) (LU Decomposition) Let $A \in \text{GL}_n(\mathbb{R})$ with all its principal minors non-zero.

(a) There exist unique matrices $L, U \in \text{GL}_n(\mathbb{R})$ of the form :

$$ L := \begin{pmatrix} 1 & & & (0) \\ l_{2,1} & 1 & & \\ \vdots & \vdots & \ddots & \\ l_{n,1} & \cdots & l_{n,n-1} & 1 \end{pmatrix}, \quad U := \begin{pmatrix} u_{1,1} & u_{1,2} & \cdots & u_{1,n} \\ & u_{22} & & \\ & & \ddots & \\ & & & u_{n,n} \end{pmatrix} $$

such that :

$$ A = LU $$

This decomposition is called **the LU decomposition of A**.
(b) We have :

$$A^{-1} = U^{-1}L^{-1}.$$

(ii) (PLU Decomposition) Let $A \in GL_n(\mathbb{R})$.

(a) There exist a permutation matrix $P \in GL_n(\mathbb{R})$ and two matrices $L, U \in GL_n(\mathbb{R})$ of the form :

$$L := \begin{pmatrix} 1 & & & & (0) \\ l_{2,1} & 1 & & & \\ \vdots & \ddots & \ddots & & \\ l_{n,1} & \cdots & l_{n,n-1} & 1 & \end{pmatrix}, \quad U := \begin{pmatrix} u_{1,1} & u_{1,2} & \cdots & u_{1,n} \\ & u_{22} & & \vdots \\ & & \ddots & \vdots \\ & & & u_{n,n} \end{pmatrix}$$

such that :

$$A = PLU$$

This decomposition is called a **PLU decomposition** of A.

(b) We have :

$$A^{-1} = U^{-1}L^{-1}P^{-1}.$$

**Proof.** Let's prove the result for the case $n := 3$. Let :

$$A := \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} \in GL_3(\mathbb{R}).$$

(i) Let the three principal minors of A be :

$$\mu_1 := \mu_1(A) = a_{11}$$

$$\mu_2 := \mu_2(A) = \begin{vmatrix} a_{11} & a_{12} \\ a_{21} & a_{22} \end{vmatrix} = a_{11}a_{22} - a_{12}a_{21}$$

$$\mu_3 := \mu_3(A) = \det(A)$$

Let :

$$L := \begin{pmatrix} 1 & 0 & 0 \\ l_{21} & 1 & 0 \\ l_{31} & l_{32} & 1 \end{pmatrix}, \quad U := \begin{pmatrix} u_{11} & u_{12} & u_{13} \\ 0 & u_{22} & u_{23} \\ 0 & 0 & u_{33} \end{pmatrix} \in \mathcal{M}_3(\mathbb{R}).$$

By identification, we have :

$$A = LU$$

$$\iff \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ l_{21} & 1 & 0 \\ l_{31} & l_{32} & 1 \end{pmatrix} \begin{pmatrix} u_{11} & u_{12} & u_{13} \\ 0 & u_{22} & u_{23} \\ 0 & 0 & u_{33} \end{pmatrix}$$

$$\iff \begin{cases} \text{1st row :} & a_{11} = u_{11}, a_{12} = u_{12}, a_{13} = u_{13} \\ \text{2nd row :} & a_{21} = l_{21}u_{11}, a_{22} = l_{21}u_{12} + u_{22}, a_{23} = l_{21}u_{13} + u_{23} \\ \text{3rd row :} & a_{31} = l_{31}u_{11}, a_{32} = l_{31}u_{12} + l_{32}u_{22}, a_{33} = l_{31}u_{13} + l_{32}u_{23} + u_{33} \\ \text{determinant :} & \mu_3 = \det(A) = u_{11}u_{22}u_{33} \end{cases}$$
$$ \iff \mu_1, \mu_2 \neq 0 \wedge \left\{ \begin{array}{l} u_{11} = a_{11} = \mu_1, \ u_{12} = a_{12}, \ u_{13} = a_{13} \\ l_{21} = \frac{a_{21}}{u_{11}} = \frac{a_{21}}{\mu_1} \\ u_{22} = a_{22} - l_{21}u_{12} = a_{22} - \frac{a_{21}a_{12}}{\mu_1} = \frac{\mu_2}{\mu_1} \\ u_{23} = a_{23} - l_{21}u_{13} = \frac{\Delta_{32}}{\mu_1} \\ l_{31} = \frac{a_{31}}{u_{11}} = \frac{a_{31}}{\mu_1} \\ l_{32} = \frac{a_{32} - l_{31}u_{12}}{u_{11}u_{22}} = \frac{\Delta_{23}}{\mu_2} \\ u_{33} = \frac{\mu_3}{u_{11}u_{22}} = \frac{\mu_3}{\mu_2} \end{array} \right. \\ \iff \mu_1, \mu_2 \neq 0 \wedge L = \begin{pmatrix} 1 & 0 & 0 \\ a_{21}/\mu_1 & 1 & 0 \\ a_{31}/\mu_1 & \Delta_{23}/\mu_2 & 1 \end{pmatrix}, \quad U = \begin{pmatrix} \mu_1 & a_{12} & a_{13} \\ 0 & \mu_2/\mu_1 & \Delta_{32}/\mu_1 \\ 0 & 0 & \mu_3/\mu_2 \end{pmatrix} $$

Thus, by reasoning by equivalence, we have the existence and uniqueness of the LU decomposition.

(ii) It is sufficient to show that there exists a permutation matrix $P'$ such that $P'A$ has all its principal minors non-zero, because then $P'A$ can be written as $P'A = LU$ as in case (i) i.e. :

$$ A = P'^{-1}LU = PLU. $$

with $P := (P')^{-1}$, which is also a permutation matrix.

Since $A$ is invertible, its first column is non-zero. Therefore, there exists an index $i_0 \in \{1, 2, 3\}$ such that $a_{i_0, 1} \neq 0$. Let :

$$ P' := P_{1,i_0}. $$

Let $A' := P'A$. The matrix $A'$ is of the type :

$$ A' := \begin{pmatrix} a_{i_0, 1} & a'_{12} & a'_{13} \\ a'_{21} & a'_{22} & a'_{23} \\ a'_{31} & a'_{32} & a'_{33} \end{pmatrix} $$

Since $A'$ is invertible (because $A$ and $P'$ are invertible), the first two columns of $A'$ are not collinear. Therefore, one of the following two determinants is non-zero (if both were zero, the two columns would be collinear, so $A'$ would not be invertible) :

$$ \alpha := \begin{vmatrix} a_{i_0, 1} & a'_{12} \\ a'_{21} & a'_{22} \end{vmatrix}, \quad \beta := \begin{vmatrix} a_{i_0, 1} & a'_{12} \\ a'_{31} & a'_{32} \end{vmatrix}. $$

We then have two cases.

(1) Case $\alpha \neq 0$. Then we have :

$$ \mu_1(A') := a_{i_0, 1}, \quad \mu_2(A') := \alpha, \quad \mu_3(A') := \det(A') $$

which are all non-zero. Thus, $A'$ satisfies the conditions of (i), and by the remark made at the beginning of the proof of (ii), we have the result.

(2) Case $\alpha = 0$ and $\beta \neq 0$. Let :

$$ P'' := P_{2,3}, \quad A'' := P''A' = P''P'A. $$

The matrix $P''P$ is a permutation matrix. The matrix $A''$ is invertible and is of the type :

$$ A'' := \begin{pmatrix} a_{i_0, 1} & a'_{12} & a'_{13} \\ a'_{31} & a'_{32} & a'_{33} \\ a'_{21} & a'_{22} & a'_{23} \end{pmatrix} $$
Then we have :

$$ \mu_1(A'') := a_{i_0,1} , \quad \mu_2(A'') := \beta , \quad \mu_3(A'') := \det(A'') $$

which are all non-zero. Thus, $A''$ also satisfies the conditions of (i), and by the remark made at the beginning of the proof of (ii), we have the result.

An LU decomposition gives a PLU decomposition by taking $P := I_n$. As we will see in the following example, the PLU decomposition is not necessarily unique, i.e. there is not necessarily a single permutation matrix $P$ such that $PA$ has all its principal minors non-zero.

### Example 2.1.3

Let :

$$ A := \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix}. $$

We will find two distinct PLU decompositions for $A$.

(1) The two principal minors of $A$ are :

$$ \mu_1(A) = 1, \quad \mu_2(A) = \det(A) = 1 - 4 = -3. $$

Since they are non-zero, $A$ has a unique LU decomposition. Let :

$$ L := \begin{pmatrix} 1 & 0 \\ l & 1 \end{pmatrix}, \quad U := \begin{pmatrix} u & v \\ 0 & w \end{pmatrix}. $$

We have :

$$ \begin{align*} A = LU &\iff \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ l & 1 \end{pmatrix} \begin{pmatrix} u & v \\ 0 & w \end{pmatrix} \\ &\iff \left\{ \begin{array}{l} 1 = u, 2 = v \\ 2 = lu, \det(A) = -3 = uw \end{array} \right. \\ &\iff \left\{ \begin{array}{l} u = 1, v = 2 \\ l = 2, w = -3 \end{array} \right. \\ &\iff L = \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix}, \quad U = \begin{pmatrix} 1 & 2 \\ 0 & -3 \end{pmatrix} \end{align*} $$

(2) Let :

$$ P := \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix}, \quad A' := PA = \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 1 & 2 \\ 2 & 1 \end{pmatrix} = \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix}. $$

The two principal minors of $A'$ are :

$$ \mu_1(A') = 2, \quad \mu_2(A') = \det(A') = 4 - 1 = 3. $$

Since they are non-zero, $A'$ has a unique LU decomposition. Let :

$$ L' := \begin{pmatrix} 1 & 0 \\ l' & 1 \end{pmatrix}, \quad U' := \begin{pmatrix} u' & v' \\ 0 & w' \end{pmatrix}. $$

We have :

$$ A' = L'U' \iff \begin{pmatrix} 2 & 1 \\ 1 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 0 \\ l' & 1 \end{pmatrix} \begin{pmatrix} u' & v' \\ 0 & w' \end{pmatrix} $$
$$ \begin{align*} & \iff \begin{cases} 2 = u', & 1 = v' \\ 1 = l'u', & \det(A') = 3 = u'w' \end{cases} \\ & \iff \begin{cases} u' = 2, & v' = 1 \\ l' = 1/2, & w' = 3/2 \end{cases} \\ & \iff L' = \begin{pmatrix} 1 & 0 \\ 1/2 & 1 \end{pmatrix}, \quad U' = \begin{pmatrix} 2 & 1 \\ 0 & 3/2 \end{pmatrix} \end{align*} $$

In conclusion, we found two distinct PLU decompositions for A :

$$ \begin{align*} A &= LU \\ &= I_2LU \\ &= \begin{pmatrix} 1 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 2 & 1 \end{pmatrix} \begin{pmatrix} 1 & 2 \\ 0 & -3 \end{pmatrix} \\ A &= P^{-1}A'LU \\ &= P'L'U' \\ &= \begin{pmatrix} 0 & 1 \\ 1 & 0 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 1/2 & 1 \end{pmatrix} \begin{pmatrix} 2 & 1 \\ 0 & 3/2 \end{pmatrix} \end{align*} $$

with $P' := P^{-1}$.

We then obtain the following fundamental result.

### Corollary 2.1.4

Let $A \in \text{GL}_n(\mathbb{R})$. Then A can be written as the product of at most :

$$n^2 + n - 1$$

elementary matrices.

**Proof.** The matrix A admits a PLU decomposition by Proposition 2.1.2, i.e. there exists a permutation matrix $P \in \text{GL}_n(\mathbb{R})$ and two matrices $L, U \in \text{GL}_n(\mathbb{R})$ of the form :

$$ L := \begin{pmatrix} 1 & & & (0) \\ l_{2,1} & 1 & & \\ \vdots & \vdots & \ddots & \\ l_{n,1} & \cdots & l_{n,n-1} & 1 \end{pmatrix}, \quad U := \begin{pmatrix} u_{1,1} & u_{1,2} & \cdots & u_{1,n} \\ & u_{22} & & \vdots \\ & & \ddots & \\ (0) & & & u_{n,n} \end{pmatrix} $$

such that :

$$A = PLU$$

By Proposition A.6.3, the matrices $L$ and $U$ decompose into a product of elementary matrices, i.e. there exist $r \in \{0, \dots, n(n-1)/2\}$, $s \in \{0, \dots, n(n+1)/2\}$, and $B_1, \dots, B_r, B_{r+1}, \dots, B_{r+s} \in \mathcal{E}_n(\mathbb{R})$ such that :

$$L = B_r \cdots B_1, \quad U = B_{r+s} \cdots B_{r+1}.$$

By point (i,b) of Lemma A.5.1, there exists an integer $t \in \{0, \dots, n-1\}$ and transposition matrices $B_{r+s+t}, \dots, B_{r+s+1} \in \mathcal{E}_n(\mathbb{R})$ such that :

$$P_{\sigma} = B_{r+s+1} \cdots B_{r+s+t}$$
Thus we have :

$$A = B_{r+s+t} \cdots B_1.$$

and :

$$r+s+t \le \frac{n(n-1)}{2} + \frac{n(n+1)}{2} + n-1 = n^2 + n - 1.$$

□

## 2.2 Example

Let's compute the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

by decomposing A using the PLU decomposition or the LU decomposition. Let $A := (a_{ij})_{ij}$. The principal minors of A are :

$$\mu_1(A) = a_{11} = 2$$

$$\mu_2(A) = \begin{vmatrix} 2 & -2 \\ 2 & -3 \end{vmatrix} = -2$$

$$\mu_3(A) = \det(A) = -3.$$

They are all non-zero. So we are in condition (i) : the matrix A has a unique LU decomposition.

* Calculating L and U. Let

$$L := \begin{pmatrix} 1 & 0 & 0 \\ l_{21} & 1 & 0 \\ l_{31} & l_{32} & 1 \end{pmatrix}, \quad U := \begin{pmatrix} u_{11} & u_{12} & u_{13} \\ 0 & u_{22} & u_{23} \\ 0 & 0 & u_{33} \end{pmatrix} \in \mathcal{M}_3(\mathbb{R})$$

We have :

$$A = LU$$

$$\iff \begin{pmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ l_{21} & 1 & 0 \\ l_{31} & l_{32} & 1 \end{pmatrix} \begin{pmatrix} u_{11} & u_{12} & u_{13} \\ 0 & u_{22} & u_{23} \\ 0 & 0 & u_{33} \end{pmatrix}$$

$$\iff \begin{cases} \text{1st row : } a_{11} = u_{11}, a_{12} = u_{12}, a_{13} = u_{13} \\ \text{2nd row : } a_{21} = l_{21}u_{11}, a_{22} = l_{21}u_{12} + u_{22}, a_{23} = l_{21}u_{13} + u_{23} \\ \text{3rd row : } a_{31} = l_{31}u_{11}, a_{32} = l_{31}u_{12} + l_{32}u_{22}, a_{33} = l_{31}u_{13} + l_{32}u_{23} + u_{33} \end{cases}$$

$$\iff \begin{cases} u_{11} = 2, u_{12} = -2, u_{13} = 1 \\ l_{21} = a_{21}/u_{11} = 1 \\ u_{22} = a_{22} - l_{21}u_{12} = -1 \\ u_{23} = a_{23} - l_{21}u_{13} = 1 \\ l_{31} = a_{31}/u_{11} = -1/2 \\ l_{32} = (a_{32} - l_{31}u_{12})/u_{22} = -1 \\ u_{33} = a_{33} - l_{31}u_{13} - l_{32}u_{23} = 3/2 \end{cases}$$

$$\iff L = \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ -1/2 & -1 & 1 \end{pmatrix}, \quad U = \begin{pmatrix} 2 & -2 & 1 \\ 0 & -1 & 1 \\ 0 & 0 & 3/2 \end{pmatrix}$$
• **Calculating $L^{-1}$ and $U^{-1}$**. The matrices $L$ and $U$ are invertible since they are triangular with no zeros on their diagonals. We then have :

$$A^{-1} = U^{-1}L^{-1}$$

We can find the inverses of $L$ and $U$ using one of the methods presented in the sections of the text. There is also a direct expression in the appendix for the case of a triangular matrix (see subsection A.6). We directly have :

$$L^{-1} = \begin{pmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ -1/2 & 1 & 1 \end{pmatrix}, \quad U^{-1} = \begin{pmatrix} 1/2 & -1 & 1/3 \\ 0 & -1 & 2/3 \\ 0 & 0 & 2/3 \end{pmatrix}.$$

• **Calculating $A^{-1}$**. Thus we have :

$$\begin{aligned} A^{-1} &= U^{-1}L^{-1} \\ &= \begin{pmatrix} 1/2 & -1 & 1/3 \\ 0 & -1 & 2/3 \\ 0 & 0 & 2/3 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -1 & 1 & 0 \\ -1/2 & 1 & 1 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{aligned}$$

## 2.3 Example of Decomposition into Elementary Matrices

Let us decompose :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

into elementary matrices of size 3. We have $A = LU$ with :

$$L = \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ -1/2 & -1 & 1 \end{pmatrix}, \quad U = \begin{pmatrix} 2 & -2 & 1 \\ 0 & -1 & 1 \\ 0 & 0 & 3/2 \end{pmatrix}.$$

By Propositions A.6.1 and A.6.2, we can decompose $L$ and $U$ into :

$$\begin{aligned} L &= \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ -1/2 & -1 & 1 \end{pmatrix} = \mathbf{T}_{2,1}(1)\mathbf{T}_{3,1}(-1/2)\mathbf{T}_{3,2}(-1) \\ &= \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1/2 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -1 & 1 \end{pmatrix} \\ U &= \begin{pmatrix} 2 & -2 & 1 \\ 0 & -1 & 1 \\ 0 & 0 & 3/2 \end{pmatrix} = \mathbf{D}_1(2)\mathbf{D}_2(-1)\mathbf{D}_3(3/2)\mathbf{T}_{1,2}(-1)\mathbf{T}_{1,3}(1/2)\mathbf{T}_{2,3}(-1) \\ &= \begin{pmatrix} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 3/2 \end{pmatrix} \begin{pmatrix} 1 & -1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 1/2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{pmatrix} \end{aligned}$$
Thus, we have :

$$
\begin{align*}
A &= LU \\
&= \mathbf{T}_{2,1}(1)\mathbf{T}_{3,1}(-1/2)\mathbf{T}_{3,2}(-1)\mathbf{D}_{1}(2)\mathbf{D}_{2}(-1)\mathbf{D}_{3}(3/2)\mathbf{T}_{1,2}(-1)\mathbf{T}_{1,3}(1/2)\mathbf{T}_{2,3}(-1) \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 1 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -1/2 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -1 & 1 \end{pmatrix} \begin{pmatrix} 2 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & -1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \\
&\quad \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 3/2 \end{pmatrix} \begin{pmatrix} 1 & -1 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 1/2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & -1 \\ 0 & 0 & 1 \end{pmatrix}
\end{align*}
$$
# 3 Gaussian Elimination

In this section, we present the algorithm for Gaussian elimination. By acting on the rows of a linear system associated with an invertible matrix, this algorithm transforms the system into a triangular form without any zero elements on the diagonal. It then becomes easy to solve by "back-substituting" the equations, which gives us an expression for the inverse. We use the notations from Appendix B.

## 3.1 Introduction

### Proposition 3.1.1

Let $A := (a_{ij})_{ij} \in GL_n(\mathbb{R})$, $B \in \mathbb{R}^n$, and consider the system :

$$ (E) : AX = B $$

with the unknown $X \in \mathbb{R}^n$. To compute $A^{-1}$, we can solve the system (E) by operating on the rows, and we have the equivalence for $X \in \mathbb{R}^n$ :

$$ X \in \mathrm{Sol}(E) \iff AX = B \iff X = A^{-1}B. $$

This method is called the **Gaussian elimination method** or **Gauss-Jordan elimination**.

**Proof.** By Corollary 2.1.4, there exist $r \in \mathbb{N}$ and elementary matrices $B_1, \dots, B_r \in \mathcal{E}_n(\mathbb{R})$ such that :

$$ A = B_1 \cdots B_r $$

i.e. we have :

$$ B_r^{-1} \cdots B_1^{-1} A = I_n. $$

And thus we have :

$$ A^{-1} = B_r^{-1} \cdots B_1^{-1}. $$

According to Appendix B, multiplying a matrix on the left by an elementary matrix $B_i^{-1}$ is equivalent to acting on the rows of the matrix. Let us define for all $i \in \{1, \dots, r\}$ :

$$ \mathrm{Op}_i := \mathrm{Op}_L(B_i^{-1}). $$

Let $X, B \in \mathbb{R}^n$. Then, by acting on the system :

$$ AX = B $$

and successively performing the row operations :

$\mathrm{Op}_1$ then $\mathrm{Op}_2$ then $\cdots$ $\mathrm{Op}_r$

i.e. by multiplying on the left by :

$B_1^{-1}$ then $B_2^{-1}$ then $\cdots$ $B_r^{-1}$

we have :

$$ \begin{align*} X \in \mathrm{Sol}(E) &\iff AX = B \\ &\iff B_r^{-1} \cdots B_1^{-1} AX = B_r^{-1} \cdots B_1^{-1} B \\ &\iff X = A^{-1}B \end{align*} $$

Thus, the result follows. $\square$
Let us define :

$$A := (a_{ij})_{ij}, \quad A^{-1} := (c_{ij})_{ij} \in \mathrm{GL}_n(\mathbb{R}), \quad X := (x_i)_i, \quad B := (b_i)_i \in \mathbb{R}^n.$$

Then, the system (E) is equivalent to :

$$(E) : \begin{cases} a_{1,1}x_1 + a_{1,2}x_2 + \cdots + a_{1,n}x_n = b_1 \\ a_{2,1}x_1 + a_{2,2}x_2 + \cdots + a_{2,n}x_n = b_2 \\ \vdots \\ a_{n,1}x_1 + a_{n,2}x_2 + \cdots + a_{n,n}x_n = b_n \end{cases}$$

By acting on the rows of (E) using the three elementary row operations (see Appendix B), we obtain that the system (E) is equivalent to :

$$\left\{ \begin{aligned} x_1 &= c_{1,1}b_1 + c_{1,2}b_2 + \cdots + c_{1,n}b_n \\ x_2 &= c_{2,1}b_1 + c_{2,2}b_2 + \cdots + c_{2,n}b_n \\ \vdots \\ x_n &= c_{n,1}b_1 + c_{n,2}b_2 + \cdots + c_{n,n}b_n \end{aligned} \right.$$

## 3.2 Example

Let us calculate the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

by performing row operations using Gaussian elimination. Let :

$$\begin{pmatrix} x \\ y \\ z \end{pmatrix}, \begin{pmatrix} a \\ b \\ c \end{pmatrix} \in \mathbb{R}^3.$$

We have :

$$\begin{align*} A \begin{pmatrix} x \\ y \\ z \end{pmatrix} &= \begin{pmatrix} a \\ b \\ c \end{pmatrix} &\iff & \left\{ \begin{array}{lclcl} 2x & - & 2y & + & z & = & a \\ 2x & - & 3y & + & 2z & = & b \\ -x & + & 2y & & & = & c \end{array} \right. & & & \\[1em] &\iff & \left\{ \begin{array}{lclcl} -x & + & 2y & & = & c & L_1 \leftrightarrow L_3 \\ 2x & - & 3y & + & 2z & = & b \\ 2x & - & 2y & + & z & = & a \end{array} \right. & & & \\[1em] &\iff & \left\{ \begin{array}{lclcl} -x & + & 2y & & = & c \\ y & + & 2z & = & b & + & 2c & L_2 \leftarrow L_2 + 2L_1 \\ 2y & + & z & = & a & + & 2c & L_3 \leftarrow L_3 + 2L_1 \end{array} \right. & & & \\[1em] &\iff & \left\{ \begin{array}{lclcl} -x & + & 2y & & = & c \\ 2y & + & z & == & b & + & 2c \\ & & & & & & & -3z & = & a & -2b & -2c & L_3 \leftarrow L_3 - 2L_2 \end{array} \right. & & & \\[1em] &\iff & \left\{ \begin{array}{lclcl} x & = & 2y - c \\ y & = & (b + 2c - z)/2 \\ z & = & (-a + 2b + 2c)/3 \end{array} \right. & & & \\[1em] &\iff & \left\{ \begin{array}{lclcl} x & = & (4a - 2b + c)/3 \\ y & = & (2a - b + 2c)/3 \\ z & = & (-a + 2b + 2c)/3 \end{array} \right. & & & \end{align*}$$
$$ \iff \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \begin{pmatrix} a \\ b \\ c \end{pmatrix} $$

Therefore, we have :

$$ A^{-1} = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}. $$
# 4 Operation on Rows

We use the notations from Appendix B. The method is similar to the Gaussian elimination method from the previous section. The only difference is that instead of acting on the rows of a linear system, we act on the rows of a block matrix.

## 4.1 Introduction

### Proposition 4.1.1

Let $A \in \text{GL}_n(\mathbb{R})$. There exists $r \in \mathbb{N}_*$ and elementary row operation matrices $B_1, \dots, B_r$ of size $n$ such that by acting on the block matrix $(A | I_n)$, we have :

$$B_r \cdots B_1 (A | I_n) = (I_n | A^{-1})$$

i.e. we have :

$$(B_r \cdots B_1 A | B_r \cdots B_1) = (I_n | A^{-1}).$$

Therefore, we have :

$$A^{-1} = B_r \cdots B_1.$$

**Proof.** The proof is similar to that of Proposition 3.1.1. By Corollary 2.1.4, there exists $r \in \mathbb{N}$ and $B_1, \dots, B_r \in \mathcal{E}_n(\mathbb{R})$ such that :

$$A = B_1 \cdots B_r$$

i.e. we have :

$$B_r^{-1} \cdots B_1^{-1} A = I_n.$$

And thus we have :

$$A^{-1} = B_r^{-1} \cdots B_1^{-1}.$$

By Appendix B, multiplying a matrix on the left by a matrix $B_i$ is equivalent to acting on the rows of the matrix. At each step, we have :

$$ \begin{array}{l} (A | I_n) \sim_L B_1 (A | I_n) = (B_1 A | B_1) \\ (B_1 A | B_1) \sim_L B_2 (B_1 A | B_1) = (B_2 B_1 A | B_2 B_1) \\ \vdots \\ (B_{r-1} \cdots B_1 A | B_{r-1} \cdots B_1) \sim_L B_r (B_{r-1} \cdots B_1 A | B_{r-1} \cdots B_1) \\ = (B_r \cdots B_1 A | B_r \cdots B_1) = (I_n | A^{-1}). \end{array} $$

Hence the result.

□

## 4.2 Example

Let's compute the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

by operating on the rows of the block matrix $(A | I_3)$. We have successively :

$$(A | I_3) = \begin{pmatrix} 2 & -2 & 1 & 1 & 0 & 0 \\ 2 & -3 & 2 & 0 & 1 & 0 \\ -1 & 2 & 0 & 0 & 0 & 1 \end{pmatrix}$$
$$ \sim_L \begin{pmatrix} -1 & 2 & 0 & 0 & 0 & 1 \\ 2 & -3 & 2 & 0 & 1 & 0 \\ 2 & -2 & 1 & 1 & 0 & 0 \end{pmatrix} \qquad L_1 \leftrightarrow L_3 $$

$$ \sim_L \begin{pmatrix} -1 & 2 & 0 & 0 & 0 & 1 \\ 0 & 1 & 2 & 0 & 1 & 2 \\ 0 & 2 & 1 & 1 & 0 & 2 \end{pmatrix} \qquad \begin{array}{l} L_2 \leftrightarrow L_2 + 2L_1 \\ L_3 \leftrightarrow L_3 + 2L_1 \end{array} $$

$$ \sim_L \begin{pmatrix} -1 & 2 & 0 & 0 & 0 & 1 \\ 0 & 1 & 2 & 0 & 1 & 2 \\ 0 & 0 & -3 & 1 & -2 & -2 \end{pmatrix} \qquad L_3 \leftrightarrow L_3 - 2L_2 $$

$$ \sim_L \begin{pmatrix} -1 & 2 & 0 & 0 & 0 & 1 \\ 0 & 3 & 0 & 2 & -1 & 2 \\ 0 & 0 & -3 & 1 & -2 & -2 \end{pmatrix} \qquad L_2 \leftrightarrow 3L_2 + 2L_3 $$

$$ \sim_L \begin{pmatrix} -3 & 0 & 0 & -4 & 2 & -1 \\ 0 & 3 & 0 & 2 & -1 & 2 \\ 0 & 0 & -3 & 1 & -2 & -2 \end{pmatrix} \qquad L_1 \leftrightarrow 3L_1 - 2L_2 $$

$$ \sim_L \begin{pmatrix} 1 & 0 & 0 & 4/3 & -2/3 & 1/3 \\ 0 & 1 & 0 & 2/3 & -1/3 & 2/3 \\ 0 & 0 & 1 & -1/3 & 2/3 & 2/3 \end{pmatrix} \qquad \begin{array}{l} L_1 \leftrightarrow -1/3.L_1 \\ L_2 \leftrightarrow 1/3.L_2 \\ L_3 \leftrightarrow -1/3.L_3 \end{array} $$

Thus, we have :

$$ A^{-1} = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}. $$

## 4.3 Product of the Matrices Associated with Elementary Row Operations

Assign to each operation (elementary or otherwise) on the rows its matrix (see section B for the notations used) :

$$ \mathrm{Mat}(L_1 \leftrightarrow L_3) = \mathbf{P}_{1,3} =: B_1 $$

$$ \mathrm{Mat}(L_2 \leftrightarrow L_2 + 2L_1) = \mathbf{T}_{2,1}(2) =: B_2 $$

$$ \mathrm{Mat}(L_3 \leftrightarrow L_3 + 2L_1) = \mathbf{T}_{3,1}(2) =: B_3 $$

$$ \mathrm{Mat}(L_3 \leftrightarrow L_3 - 2L_2) = \mathbf{T}_{3,1}(-2) =: B_4 $$

$$ \begin{align*} \mathrm{Mat}(L_2 \leftrightarrow 3L_2 + 2L_3) &= \mathrm{Mat}(L_2 \leftrightarrow 3L_2) \mathrm{Mat}(L_2 \leftrightarrow L_2 + 2/3.L_3) \\ &= \mathbf{D}_2(3)\mathbf{T}_{2,3}(2/3) =: B_6B_5 \end{align*} $$

$$ \begin{align*} \mathrm{Mat}(L_1 \leftrightarrow 3L_1 - 2L_2) &= \mathrm{Mat}(L_1 \leftrightarrow 3L_1) \mathrm{Mat}(L_1 \leftrightarrow L_1 - 2/3.L_2) \\ &= \mathbf{D}_1(3)\mathbf{T}_{1,2}(-2/3) =: B_8B_7 \end{align*} $$

$$ \mathrm{Mat}(L_1 \leftrightarrow -1/3.L_1) = \mathbf{D}_1(-1/3) =: B_9 $$

$$ \mathrm{Mat}(L_2 \leftrightarrow 1/3.L_2) = \mathbf{D}_2(1/3) =: B_{10} $$

$$ \mathrm{Mat}(L_3 \leftrightarrow -1/3.L_3) = \mathbf{D}_3(-1/3) =: B_{11} $$

Let us show that the product of these elementary operations yields the inverse of A. We have :<fnref n="3" />

$$ \begin{align*} B_{11}B_{10}B_{9}B_{8}B_{7}B_{6}B_{5}B_{4}B_{3}B_{2}B_{1} \\ = \mathbf{D}_{3}(-1/3)\mathbf{D}_{2}(1/3)\mathbf{D}_{1}(-1/3)\mathbf{D}_{1}(3)\mathbf{T}_{1,2}(-2/3)\mathbf{D}_{2}(3)\mathbf{T}_{2,3}(2/3) \\ \mathbf{T}_{3,2}(-2)\mathbf{T}_{3,1}(2)\mathbf{T}_{2,1}(2)\mathbf{P}_{1,3} \end{align*} $$

<footnote n="3">3. We separate by the symbol ● the multiplications done together.</footnote>
$$
\begin{align*}
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & -1/3 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/3 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} -1/3 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \cdot \begin{pmatrix} 3 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & -2/3 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \cdot \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 2/3 \\ 0 & 0 & 1 \end{pmatrix} \cdot \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & 0 & 1 \end{pmatrix} \cdot \begin{pmatrix} 1 & 0 & 0 \\ 2 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} \\
&= \begin{pmatrix} -1/3 & 0 & 0 \\ 0 & 1/3 & 0 \\ 0 & 0 & -1/3 \end{pmatrix} \begin{pmatrix} 3 & -2 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \cdot \begin{pmatrix} 1 & 0 & 0 \\ 0 & 3 & 2 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 2 & -2 & 1 \end{pmatrix} \cdot \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 2 \\ 1 & 0 & 0 \end{pmatrix} \\
&= \begin{pmatrix} -1 & 2/3 & 0 \\ 0 & 1/3 & 0 \\ 0 & 0 & -1/3 \end{pmatrix} \cdot \begin{pmatrix} 1 & 0 & 0 \\ 4 & -1 & 2 \\ 2 & -2 & 1 \end{pmatrix} \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 2 \\ 1 & 0 & 0 \end{pmatrix} \\
&= \begin{pmatrix} -1 & 2/3 & 0 \\ 0 & 1/3 & 0 \\ 0 & 0 & -1/3 \end{pmatrix} \begin{pmatrix} 0 & 0 & 1 \\ 2 & -1 & 2 \\ 1 & -2 & -2 \end{pmatrix} \\
&= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \\
&= A^{-1}
\end{align*}
$$
# 5 Operations on Columns

We use the notations from Appendix C. The method is similar to the previous section 4. This time, we act on the columns of a block matrix.

## 5.1 Introduction

### Proposition 5.1.1

Let $A \in \text{GL}_n(\mathbb{R})$. There exist $r \in \mathbb{N}_*$ and elementary column operation matrices $B_1, \dots, B_r$ of size $n$ such that by acting on the block matrix $(\frac{A}{I_n})$, we have :

$$ \left(\frac{A}{I_n}\right) B_1 \cdots B_r = \left(\frac{I_n}{A^{-1}}\right) $$

i.e., we have :

$$ \left( \frac{AB_1 \cdots B_r}{B_1 \cdots B_r} \right) = \left( \frac{I_n}{A^{-1}} \right). $$

Thus, we have :

$$ A^{-1} = B_1 \cdots B_r. $$

**Proof.** Let us consider the block matrix :

$$ (A' | I_n) := (A^T | I_n) = \left(\frac{A}{I_n}\right)^T. $$

We are in the case of Proposition 4.1.1 with the block matrix $(A' | I_n)$. There exist $r \in \mathbb{N}_*$ and elementary row operation matrices $B'_1, \dots, B'_r$ of size $n$ such that by acting on the block matrix $(A' | I_n)$, we have :

$$ B'_r \cdots B'_1 (A' | I_n) = (I_n | (A')^{-1}) $$

i.e., we have :

$$ (B'_r \cdots B'_1 A' | B'_r \cdots B'_1) = (I_n | (A')^{-1}). $$

Thus :

$$ A'^{-1} = B'_r \cdots B'_1. $$

By taking the transpose and defining for all $i \in \{1, \dots, r\}$ :

$$ B_i := (B'_i)^T, $$

we then have at each step :

$$ \begin{align*} \left(\frac{A}{I_n}\right) &\sim_C \left(\frac{A}{I_n}\right) B_1 = \left(\frac{AB_1}{B_1}\right) \\ \left(\frac{AB_1}{B_1}\right) &\sim_C \left(\frac{AB_1}{B_1}\right) B_2 = \left(\frac{AB_1 B_2}{B_1 B_2}\right) \\ &\vdots \\ \left(\frac{AB_1 \cdots B_{r-1}}{B_1 \cdots B_{r-1}}\right) &\sim_C \left(\frac{AB_1 \cdots B_{r-1}}{B_1 \cdots B_{r-1}}\right) B_r \\ &= \left(\frac{AB_1 \cdots B_r}{B_1 \cdots B_r}\right) = \left(\frac{I_n}{A^{-1}}\right) \end{align*} $$

□
## 5.2 Example

Calculate the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

by performing operations on the columns of the block matrix $(\frac{A}{I_3})$. We proceed successively :

$$\left(\frac{A}{I_3}\right) = \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

$$\sim_C \begin{pmatrix} 1 & -2 & 2 \\ 2 & -3 & 2 \\ 0 & 2 & -1 \end{pmatrix}$$

$$C_1 \leftrightarrow C_3$$

$$\sim_C \begin{pmatrix} 1 & 0 & 0 \\ 2 & 1 & -2 \\ 0 & 2 & -1 \end{pmatrix}$$

$$C_2 \leftrightarrow C_2 + 2C_1$$

$$C_3 \leftrightarrow C_3 - 2C_1$$

$$\sim_C \begin{pmatrix} 1 & 0 & 0 \\ 2 & 1 & 0 \\ 0 & 2 & 3 \end{pmatrix}$$

$$C_3 \leftrightarrow C_3 + 2C_2$$

$$\sim_C \begin{pmatrix} 1 & 0 & 0 \\ 2 & 3 & 0 \\ 0 & 0 & 3 \end{pmatrix}$$

$$C_2 \leftrightarrow 3C_2 - 2C_3$$

$$\sim_C \begin{pmatrix} 3 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 3 \end{pmatrix}$$

$$C_1 \leftrightarrow 3C_1 - 2C_2$$
$$ \sim_C \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \qquad \begin{array}{l} C_1 \leftarrow 1/3 \cdot C_1 \\ C_2 \leftarrow 1/3 \cdot C_2 \\ C_3 \leftarrow 1/3 \cdot C_3 \end{array} $$

Thus we have :

$$ A^{-1} = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}. $$

## 5.3 Product of Matrices Associated with Elementary Operations

We associate with each operation (elementary or not) on the columns its matrix (see section C for the notations used) :

$$ \mathrm{Mat}(C_1 \leftrightarrow C_3) = \mathbf{P}_{1,3} =: B_1 $$

$$ \mathrm{Mat}(C_2 \leftrightarrow C_2 + 2C_1) = \mathbf{T}_{2,1}(2)^T =: B_2 $$

$$ \mathrm{Mat}(C_3 \leftrightarrow C_3 - 2C_1) = \mathbf{T}_{3,1}(-2)^T =: B_3 $$

$$ \mathrm{Mat}(C_3 \leftrightarrow C_3 + 2C_2) = \mathbf{T}_{3,2}(2)^T =: B_4 $$

$$ \begin{aligned} \mathrm{Mat}(C_2 \leftrightarrow 3C_2 - 2C_3) &= \mathrm{Mat}(C_2 \leftrightarrow C_2 - 2/3 \cdot C_3) \mathrm{Mat}(C_2 \leftrightarrow 3C_2) \\ &= \mathbf{T}_{2,3}(-2/3)^T \mathbf{D}_2(3) =: B_5 B_6 \end{aligned} $$

$$ \begin{aligned} \mathrm{Mat}(C_1 \leftrightarrow 3C_1 - 2C_2) &= \mathrm{Mat}(C_1 \leftrightarrow C_1 - 2/3 \cdot C_2) \mathrm{Mat}(C_1 \leftrightarrow 3C_1) \\ &= \mathbf{T}_{1,2}(-2/3)^T \mathbf{D}_1(3) =: B_7 B_8 \end{aligned} $$

$$ \mathrm{Mat}(C_1 \leftrightarrow 1/3 \cdot C_1) = \mathbf{D}_1(1/3) =: B_9 $$

$$ \mathrm{Mat}(C_2 \leftrightarrow 1/3 \cdot C_2) = \mathbf{D}_2(1/3) =: B_{10} $$

$$ \mathrm{Mat}(C_3 \leftrightarrow 1/3 \cdot C_3) = \mathbf{D}_3(1/3) =: B_{11} $$

We show that the product of these elementary operations gives the inverse of A. We have :<fnref n="4" />

$$ \begin{aligned} & B_1 B_2 B_3 B_4 B_5 B_6 B_7 B_8 B_9 B_{10} B_{11} \\ &= \mathbf{P}_{1,3} \mathbf{T}_{2,1}(2)^T \mathbf{T}_{3,1}(-2)^T \mathbf{T}_{3,2}(2)^T \mathbf{T}_{2,3}(-2/3)^T \mathbf{D}_2(3) \\ & \quad \mathbf{T}_{1,2}(-2/3)^T \mathbf{D}_1(3) \mathbf{D}_1(1/3) \mathbf{D}_2(1/3) \mathbf{D}_3(1/3) \\ &= \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} \begin{pmatrix} 1 & 2 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & -2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 2 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2/3 & 1 \end{pmatrix} \\ & \quad \begin{pmatrix} 1 & 0 & 0 \\ 0 & 3 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -2/3 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 3 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1/3 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/3 & 0 \\ 0 & 0 & 1/3 \end{pmatrix} \\ & \quad \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 1 & 0 & -2 \\ 0 & 1 & 2 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -2/3 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -2 & 3 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/3 & 0 \\ 0 & 0 & 1/3 \end{pmatrix} \end{aligned} $$

<footnote n="4">4. We separate the multiplications done together by the symbol • .</footnote>
$$ = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 2 \\ 1 & 2 & 2 \end{pmatrix} \bullet \begin{pmatrix} 1 & 0 & 0 \\ -2 & 1 & 0 \\ 4/3 & -2/3 & 1/3 \end{pmatrix} $$

$$ = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 2 \\ 1 & 2 & 2 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -2 & 1 & 0 \\ 4/3 & -2/3 & 1/3 \end{pmatrix} $$

$$ = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} $$

$$ =A^{-1} $$
# 6 Cayley-Hamilton Theorem

The Cayley-Hamilton theorem states that the characteristic polynomial of a matrix is an annihilating polynomial of that matrix. Thus, in the case of an invertible matrix, we find a quick method to calculate its inverse.

## 6.1 Introduction

We recall some useful results about the characteristic polynomial of a matrix<fnref n="5" />.

### Definition 6.1.1

Let $M \in \mathcal{M}_n(\mathbb{R})$. The characteristic polynomial of $M$ is defined by :

$$ \chi_M(X) := \det(M - XI_3). $$

Expanding the determinant, we find that the polynomial $\chi_M(X)$ is of degree $n$, with a leading coefficient $(-1)^n$ and a constant term $\det(M)$. Thus, there exists a polynomial $R \in \mathbb{R}[X]$ of degree $n-1$ such that :

$$ \chi_M(X) = XR(X) + \det(M). \qquad (6.1.1) $$

### Example 6.1.2

Let $M := (m_{ij})_{ij} \in \mathcal{M}_3(\mathbb{R})$. Expanding the determinant of size 3 by, for example, the first row, we have :

$$ \begin{aligned} & \chi_M(X) \\ &= \det(M - XI_3) \\ &= \begin{vmatrix} m_{11} - X & m_{12} & m_{13} \\ m_{21} & m_{22} - X & m_{23} \\ m_{31} & m_{32} & m_{33} - X \end{vmatrix} \\ &= (m_{11} - X) \begin{vmatrix} m_{22} - X & m_{23} \\ m_{32} & m_{33} - X \end{vmatrix} - m_{12} \begin{vmatrix} m_{21} & m_{23} \\ m_{31} & m_{33} - X \end{vmatrix} + m_{13} \begin{vmatrix} m_{21} & m_{22} - X \\ m_{31} & m_{32} \end{vmatrix} \\ &= (m_{11} - X) [(m_{22} - X)(m_{33} - X) - m_{32}m_{23}] \\ & \quad - m_{12} [m_{21}(m_{33} - X) - m_{31}m_{23}] \\ & \quad + m_{13} [m_{21}m_{32} - m_{31}(m_{22} - X)] \\ &= -X^3 + (m_{11} + m_{11} + m_{11}) X^2 \\ & \quad + (m_{12}m_{21} - m_{11}m_{22} + m_{13}m_{31} + m_{23}m_{32} - m_{11}m_{33} - m_{22}m_{33}) X \\ & \quad - m_{13}m_{22}m_{31} + m_{12}m_{23}m_{31} + m_{13}m_{21}m_{32} - m_{11}m_{23}m_{32} - m_{12}m_{21}m_{33} + m_{11}m_{22}m_{33} \\ &= -X^3 + \mathrm{Tr}(M)X^2 - c_1 X + \det(M) \end{aligned} $$

where :

$$ c_1 := m_{12}m_{21} - m_{11}m_{22} + m_{13}m_{31} + m_{23}m_{32} - m_{11}m_{33} - m_{22}m_{33}. $$

Thus we have :

$$ \chi_M(X) = XR(X) + \det(M) $$

with :

$$ R(X) := -X^2 + \mathrm{Tr}(M)X - c_1. $$

<footnote n="5">5. One could have defined $\chi_M$ as $\chi_M(X) := \det(XI_n - M)$.</footnote>
### Proposition 6.1.3: Cayley-Hamilton Theorem

Let $M \in \mathcal{M}_n(\mathbb{R})$. We have :

$$ \chi_M(M) = 0. $$

**Proof.** Let us prove the result for $n := 3$. Set :

$$ N := \mathrm{Com}(M - XI_3)^T. $$

and :

$$ -X^3 + c_2X^2 + c_1X + c_0 := \chi_M(X) = \det(M - XI_3). $$

Expanding the determinants appearing in the adjugate of $M - XI_3$, we obtain that $N$ is a polynomial of degree 2 of the form :

$$ X^2N_2 + XN_1 + N_0 := N = \mathrm{Com}(M - XI_3)^T. $$

Then by Proposition 1.1.2 :

$$
\begin{align*}
-X^3 I_3 + c_2 X^2 I_3 + c_1 X I_3 + c_0 I_3 &= \chi(X) I_3 \\
&= \det(M - XI_3) I_3 \\
&= (M - XI_3) \mathrm{Com}(M - XI_3)^T \\
&= (M - XI_3) N \\
&= (M - XI_3) (X^2 N_2 + X N_1 + N_0) \\
&= -X^3 N_2 + X^2 (MN_2 - N_1) + X (MN_1 - N_0) + MN_0
\end{align*}
$$

By the uniqueness of the decomposition in the ring $\mathcal{M}_n(\mathbb{R})[X]$, we have :

$$
\begin{align*}
c_0 I_3 &= MN_0 \\
c_1 I_3 &= MN_1 - N_0 \\
c_2 I_3 &= MN_2 - N_1 \\
-I_3 &= -N_2
\end{align*}
$$

Thus, by multiplying the second equation by $M$, the third by $M^2$, and the fourth by $M^3$, we get :

$$
\begin{align*}
c_0 I_3 &= MN_0 \\
c_1 M &= M(MN_1 - N_0) \\
c_2 M^2 &= M^2(MN_2 - N_1) \\
-M^3 &= -M^3 N_2
\end{align*}
$$

By summing these four equations, we get by telescoping :

$$
\begin{align*}
\chi_M(M) &= -M^3 + c_2 M^2 + c_1 M + c_0 I_3 \\
&= -M^3 N_2 + M^2 (MN_2 - N_1) + M (MN_1 - N_0) + MN_0 \\
&= 0
\end{align*}
$$

We deduce the following corollary in the case of an invertible matrix.

<figure><img src="image_4.png" /></figure>
### Corollary 6.1.4

Let $A \in \text{GL}_n(\mathbb{R})$. There exists a polynomial $R \in \mathbb{R}[X]$ of degree $n-1$ such that:

$$A^{-1} = \frac{-1}{\det(A)} R(A).$$

**Proof.** By Equation (6.1.1), there exists a polynomial $R \in \mathbb{R}[X]$ of degree $n-1$ such that:

$$\chi_A(X) = XR(X) + \det(A).$$

Since $\det(A) \neq 0$, by the Cayley-Hamilton theorem:

$$0 = \chi_A(A) = AR(A) + \det(A)I_n$$

i.e., we have:

$$A^{-1} = \frac{-1}{\det(A)} R(A).$$

□

Comparing with Corollary 1.1.3, we have:

$$R(A) = -\text{Com}(A)^T.$$

## 6.2 Example

Let's calculate the inverse of:

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

using the previous corollary. The characteristic polynomial of $A$ is given by:

$$\begin{align*}
\chi_A(X) &= \det(A - XI_3) \\
&= \begin{vmatrix} 2-X & -2 & 1 \\ 2 & -3-X & 2 \\ -1 & 2-X & -X \end{vmatrix} \\
&= (2-X) \begin{vmatrix} -3-X & 2 & -2 \\ 2-X & -X & -X \end{vmatrix} - \begin{vmatrix} -2 & 1 & -2 \\ 2-X & -X & -X \end{vmatrix} \\
&= (2-X)((3-X)X - 2(2-X)) - 2(2X - (2-X)) - (-4+3-X) \\
&= -X^3 - X^2 + 5X - 3
\end{align*}$$

By the Cayley-Hamilton theorem, we have:

$$\begin{align*}
0 &= \chi_A(A) \\
&= -A^3 - A^2 + 5A - 3I_3 \\
&= A(-A^2 - A + 5I_3) - 3I_3
\end{align*}$$

i.e., we have<fnref n="6" />:

$$A^{-1} = \frac{1}{3}(-A^2 - A + 5I_3)$$

<footnote n="6">6. Thus we are in the case where $R(X) := -X^2 - X + 5$.</footnote>
$$ = \frac{1}{3} \left[ - \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} - \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} + 5 \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \right] $$

$$ = \frac{1}{3} \left[ - \begin{pmatrix} -1 & 4 & -2 \\ -4 & 9 & -4 \\ 2 & -4 & 3 \end{pmatrix} - \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} + \begin{pmatrix} 5 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 5 \end{pmatrix} \right] $$

$$ = \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} $$
# 7 Formula Derived from the Cayley-Hamilton Theorem

In the previous section, we obtained an expression for the inverse of a matrix as a polynomial in the matrix (see Corollary 6.1.4). In this section, we provide the explicit formula for this polynomial. The coefficients of this polynomial depend on the traces of the powers of the matrix.

## 7.1 Introduction

To obtain the general formula, we need some notation. For all $s \in \{0, \dots, n-1\}$, let us define :

$$ \Gamma(n, s) := \left\{ (k_1, \dots, k_{n-1}) \in \mathbb{N}^{n-1}, s + \sum_{l=1}^{n-1} lk_l = n-1 \right\}. $$

### Proposition 7.1.1

Let $A \in \text{GL}_n(\mathbb{R})$. We have :

$$ A^{-1} = \frac{1}{\det(A)} \sum_{s=0}^{n-1} A^s \sum_{(k_1, \dots, k_{n-1}) \in \Gamma(n,s)} \prod_{l=1}^{n-1} \frac{(-1)^{k_l+1}}{l^{k_l} k_l!} \text{Tr}(A^l)^{k_l}. $$

**Proof.** We provide the proof in the case $n := 3$. It will be shown after the proof that in this case, the formula simplifies to :

$$ A^{-1} = \frac{1}{\det(A)} \left( \frac{1}{2} \left[ (\text{Tr}A)^2 - \text{Tr}(A^2) \right] I_3 - A \text{Tr}(A) + A^2 \right). $$

Let us prove this formula. There exist coefficients $c_1, c_2 \in \mathbb{R}$ such that the characteristic polynomial of $A$ is :

$$ \chi_A(X) = -X^3 + c_2 X^2 + c_1 X + \det(A) $$

Let $\lambda_1, \lambda_2, \lambda_3 \in \mathbb{C}$ be the complex eigenvalues of $A$ (counted with multiplicities), i.e., we have :

$$ \chi_A(X) = -(X - \lambda_1)(X - \lambda_2)(X - \lambda_3). $$

By expanding and matching the two expressions for $\chi_A$ (which are the coefficient-root relations), we get :

$$ \begin{aligned} c_2 &= \lambda_1 + \lambda_2 + \lambda_3 \\ c_1 &= -\lambda_1\lambda_2 - \lambda_1\lambda_3 - \lambda_2\lambda_3 \end{aligned} $$

For any $k \in \mathbb{N}$, let us define :

$$ s_k := \text{Tr}(A^k) $$

Since any matrix is triangularizable over $\mathbb{C}$, there exists a matrix $P \in \text{GL}_3(\mathbb{C})$ and an upper triangular matrix :

$$ T := \begin{pmatrix} \lambda_1 & * & * \\ 0 & \lambda_2 & * \\ 0 & 0 & \lambda_3 \end{pmatrix} $$

such that :

$$ T = P^{-1}AP. $$

Since for any $k \in \mathbb{N}$, $A^k = PT^k P^{-1}$, we have :

$$ \begin{aligned} s_k &= \text{Tr}(A^k) \\ &= \text{Tr}(PT^k P^{-1}) \end{aligned} $$
$$
\begin{align*}
&= \operatorname{Tr}(T^k) \\
&= \operatorname{Tr} \begin{pmatrix} \lambda_1^k & * & * \\ 0 & \lambda_2^k & * \\ 0 & 0 & \lambda_3^k \end{pmatrix} \\
&= \lambda_1^k + \lambda_2^k + \lambda_3^k
\end{align*}
$$

By the theorem on symmetric polynomials, the scalars $s_k$ can be expressed in terms of the $c_k$, and vice versa, the scalars $c_k$ can be expressed in terms of the $s_k$. These are the Newton identities. The general formulas use the **Bell polynomials** $B_k$. In the case $n := 3$, we can find the formulas manually. We have :

$$
\begin{align*}
c_2 &= \lambda_1 + \lambda_2 + \lambda_3 \\
&= s_1 \\
&= \operatorname{Tr}(A) \\
c_1 &= -\lambda_1\lambda_2 - \lambda_1\lambda_3 - \lambda_2\lambda_3 \\
&= \frac{1}{2} \left[ -(\lambda_1 + \lambda_2 + \lambda_3)^2 + (\lambda_1^2 + \lambda_2^2 + \lambda_3^2) \right] \\
&= \frac{1}{2} \left[ -s_1^2 + s_2 \right] \\
&= \frac{1}{2} \left[ -(\operatorname{Tr}A)^2 + \operatorname{Tr}(A^2) \right]
\end{align*}
$$

By the Cayley-Hamilton theorem, we have :

$$
0 = \chi_A(A) = -A^3 + c_2 A^2 + c_1 A + \det(A) I_3
$$

i.e., we get :

$$
\begin{align*}
A^{-1} &= \frac{1}{\det(A)} (A^2 - c_2 A - c_1 I_3) \\
&= \frac{1}{\det(A)} \left( \frac{1}{2} \left[ (\operatorname{Tr} A)^2 - \operatorname{Tr}(A^2) \right] I_3 - A \operatorname{Tr}(A) + A^2 \right)
\end{align*}
$$

□

We conclude this subsection by discussing the cases $n := 2$ and $n := 3$. In particular, we prove the formula given in the proof of the previous proposition for the case $n := 3$.

**Example 7.1.2**

(1) For $n := 2$, we have :

$$
\begin{align*}
\Gamma(2,0) &= \{k_1 \in \mathbb{N}, 0 + k_1 = 1\} = \{1\} \\
\Gamma(2,1) &= \{k_1 \in \mathbb{N}, 1 + k_1 = 1\} = \{0\}
\end{align*}
$$

Thus :

$$
\begin{align*}
A^{-1} &= \frac{1}{\det(A)} \sum_{s=0}^{1} A^s \sum_{k_1 \in \Gamma(1,s)} \frac{(-1)^{k_1+1}}{1^{k_1} k_1!} \operatorname{Tr}(A^1)^{k_1} \\
&= \frac{1}{\det(A)} \left( A^0 \frac{(-1)^{1+1}}{1^1 1!} \operatorname{Tr}(A^1)^2 + A^1 \frac{(-1)^{0+1}}{1^0 0!} \operatorname{Tr}(A^1)^1 \right) \\
&= \frac{1}{\det(A)} (\operatorname{Tr}(A) I_2 - A)
\end{align*}
$$
(2) For $n := 3$, we have :

$$ \Gamma(3,0) = \{(k_1, k_2) \in \mathbb{N}^2, 0 + k_1 + 2k_2 = 2\} = \{(2,0), (0,1)\} $$

$$ \Gamma(3,1) = \{(k_1, k_2) \in \mathbb{N}^2, 1 + k_1 + 2k_2 = 2\} = \{(1,0)\} $$

$$ \Gamma(3,2) = \{(k_1, k_2) \in \mathbb{N}^2, 2 + k_1 + 2k_2 = 2\} = \{(0,0)\} $$

Thus :

$$ \begin{aligned} A^{-1} &= \frac{1}{\det(A)} \sum_{s=0}^{2} A^s \sum_{(k_1,k_2) \in \Gamma(3,s)} \prod_{l=1}^{2} \frac{(-1)^{k_l+1}}{l^{k_l} k_l!} \mathrm{Tr}(A^l)^{k_l} \\ &= \frac{1}{\det(A)} \left( A^0 \left( \frac{(-1)^{2+1}}{1^2 2!} \mathrm{Tr}(A^1)^2 \frac{(-1)^{0+1}}{2^0 0!} \mathrm{Tr}(A^2)^0 + \frac{(-1)^{0+1}}{1^0 0!} \mathrm{Tr}(A^1)^0 \frac{(-1)^{1+1}}{2^1 1!} \mathrm{Tr}(A^2)^1 \right) \right. \\ &\quad \left. + A^1 \frac{(-1)^{1+1}}{1^1 1!} \mathrm{Tr}(A^1)^1 \frac{(-1)^{0+1}}{2^0 0!} \mathrm{Tr}(A^2)^0 + A^2 \frac{(-1)^{0+1}}{1^0 0!} \mathrm{Tr}(A^1)^0 \frac{(-1)^{0+1}}{2^0 0!} \mathrm{Tr}(A^2)^0 \right) \\ &= \frac{1}{\det(A)} \left( \frac{1}{2} \left[ (\mathrm{Tr}A)^2 - \mathrm{Tr}(A^2) \right] I_3 - A \mathrm{Tr}(A) + A^2 \right) \end{aligned} $$

## 7.2 Example

Let us calculate the inverse of :

$$ A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} $$

using the previous formula in the case $n := 3$ :

$$ A^{-1} = \frac{1}{\det(A)} \left( \frac{1}{2} \left[ (\mathrm{Tr}A)^2 - \mathrm{Tr}(A^2) \right] I_3 - A \mathrm{Tr}(A) + A^2 \right) $$

Since :

$$ \mathrm{Tr}(A) = -1 $$

$$ \det(A) = -3 $$

$$ A^2 = \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} = \begin{pmatrix} -1 & 4 & -2 \\ -4 & 9 & -4 \\ 2 & -4 & 3 \end{pmatrix} $$

$$ \mathrm{Tr}(A^2) = 11 $$

we have :

$$ \begin{aligned} A^{-1} &= \frac{1}{\det(A)} \left( \frac{1}{2} \left[ (\mathrm{Tr}A)^2 - \mathrm{Tr}(A^2) \right] I_3 - A \mathrm{Tr}(A) + A^2 \right) \\ &= \frac{1}{-3} \left( -5 \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} + \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} + \begin{pmatrix} -1 & 4 & -2 \\ -4 & 9 & -4 \\ 2 & -4 & 3 \end{pmatrix} \right) \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{aligned} $$
# 8 Diagonalization and Minimal Polynomial

As we have seen, the Cayley-Hamilton theorem states that the characteristic polynomial of a matrix is an annihilating polynomial of that matrix. Moreover, the set of annihilating polynomials of a matrix forms an ideal of the principal ring $\mathbb{R}[X]$. Thus, this ideal is generated by a monic polynomial of minimal degree called the minimal polynomial of the matrix. Its degree may be the same as the characteristic polynomial, but if not, it provides a faster method (requiring fewer matrix multiplications) than the one in section 6 using the Cayley-Hamilton theorem.

## 8.1 Introduction

Let $M, N \in \mathcal{M}_n(\mathbb{K})$ with $\mathbb{K} := \mathbb{R}$ or $\mathbb{C}$. We denote :

$$ \mathrm{Ann}(M) := \{P \in \mathbb{K}[X], P(M) = 0\}. $$

The Cayley-Hamilton theorem is equivalent to the assertion :

$$ \chi_M(X) \in \mathrm{Ann}(M). $$

Since $\mathrm{Ann}(M)$ is an ideal of $\mathbb{K}[X]$ (called the **ideal of annihilating polynomials of M**) and since the ring $\mathbb{K}[X]$ is **principal** i.e. for any ideal $I$ of $\mathbb{K}[X]$, there exists a polynomial $P \in \mathbb{R}[X]$ such that $I = P\mathbb{K}[X]$, we have the following definition.

**Definition 8.1.1**

The **minimal polynomial of M** is the unique monic polynomial $\pi_M$ of $\mathrm{Ann}(M)$ (of minimal degree) such that :

$$ \mathrm{Ann}(M) = \pi_M(X)\mathbb{K}[X]. $$

Since $\chi_M(X) \in \mathrm{Ann}(M)$, the polynomial $\pi_M$ therefore divides $\chi_M$. Before stating the useful result for this section, we recall diagonalization.

**Definition 8.1.2**

(i) $M$ and $N$ are said to be **similar** if there exists $P \in \mathrm{GL}_n(\mathbb{K})$ such that :

$$ M = PNP^{-1}. $$

We then denote :

$$ M \sim N. $$

(ii) $M$ is said to be **diagonalizable** if there exists a diagonal matrix $D \in \mathcal{M}_n(\mathbb{K})$ such that :

$$ M \sim D. $$

(iii) Let $\lambda \in \mathbb{K}$. $\lambda$ is said to be an **eigenvalue of M** in $\mathbb{K}$ if there exists a non-zero vector $u \in \mathbb{K}^n$ such that $Mu = \lambda u$. $u$ is said to be an **eigenvector associated with $\lambda$**.

(a) $\mathrm{Sp}_K(M)$ denotes the set of eigenvalues of $M$ in $\mathbb{K}$. We will simply denote $\mathrm{Sp}(M) := \mathrm{Sp}_R(M)$.

(b) The eigenspace associated with the eigenvalue $\lambda$ is the $\mathbb{K}$-vector space $E_\lambda(M)$ defined by :

$$ E_\lambda(M) := \{v \in \mathbb{K}^n, Mv = \lambda v\} := \mathrm{Ker}(M - \lambda I_n). $$
For all $\lambda \in \mathbb{K}$, we have the simple equivalences :

$$
\begin{align*}
\lambda \in \mathrm{Sp}_{\mathbb{K}}(M) &\iff \exists v \in \mathbb{K}^n - \{0\}, \quad Mv = \lambda v \\
&\iff \exists v \in \mathbb{K}^n - \{0\}, \quad \forall k \in \mathbb{N}, \quad M^k v = \lambda^k v \\
&\iff \exists v \in \mathbb{K}^n - \{0\}, \quad \pi_M(\lambda)v = 0 \\
&\iff \pi_M(\lambda) = 0
\end{align*}
$$

i.e. $\lambda$ is an eigenvalue of $M$ if and only if it is a root of $\pi_M$.

In other words, if $M$ is diagonalizable, there exists a basis $(u_1, \dots, u_n)$ of $\mathbb{K}^n$ and scalars $\lambda_1, \dots, \lambda_n \in \mathbb{K}$ such that :

$$
P = (u_1 | \cdots | u_n) , \quad D = \begin{pmatrix} \lambda_1 & & (0) \\ & \ddots & \\ (0) & & \lambda_n \end{pmatrix} .
$$

Since $P$ is the transition matrix from the canonical basis $(e_1, \dots, e_n)$ to the basis $(u_1, \dots, u_n)$, for all $i \in \{1, \dots, n\}$ we have $P^{-1}u_i = e_i$ and therefore :

$$
\begin{align*}
Mu_i &= PDP^{-1}u_i \\
&= PDe_i \\
&= P\lambda_i e_i \\
&= \lambda_i u_i
\end{align*}
$$

i.e. $u_i$ is an eigenvector associated with the eigenvalue $\lambda_i$ ($u_i$ is non-zero because it is a vector in a basis). Thus, we have the equivalence between :

(i) $M$ is diagonalizable in $\mathbb{K}$;

(ii) there exists a basis of $\mathbb{K}^n$ consisting of eigenvectors of $M$.

Moreover, let $\lambda \in \mathbb{K}$. We have the equivalence between :

$$
\begin{align*}
\lambda \in \mathrm{Sp}_{\mathbb{K}}(M) &\iff \exists v \in \mathbb{K}^n - \{0\}, \quad Mv = \lambda v \\
&\iff \exists v \in \mathbb{K}^n - \{0\}, \quad (M - \lambda I_n)v = 0 \\
&\iff \det(M - \lambda I_n) \neq 0 \\
&\iff \chi_M(\lambda) = 0
\end{align*}
$$

i.e. the eigenvalues of $M$ are exactly the roots of $\chi_M$.

We then have the usual lemma.

**Lemma 8.1.3**

We have the equivalence between :

(i) $M$ is diagonalizable in $\mathbb{K}$;

(ii) we have the direct sum :

$$
\mathbb{K}^n = \bigoplus_{\lambda \in \mathrm{Sp}_{\mathbb{K}}(M)} E_{\lambda}(M).
$$

**Proof.** Let us prove the result in the case where $M$ has two distinct eigenvalues $\lambda$ and $\mu$.
Assume that $M$ is diagonalizable in $\mathbb{K}$. Let us prove that the sum is direct. Let :

$$x \in E_{\lambda}(M) \cap E_{\mu}(M)$$

i.e. we have :

$$Mx = \lambda x, \quad Mx = \mu x.$$

So we have $(\lambda - \mu)x = 0$. Since $\lambda \neq \mu$, we have $x = 0$.

Let $x \in \mathbb{K}^n$. Since $M$ is diagonalizable, there exists a basis of eigenvectors $x_1, \dots, x_k \in E_\lambda(M)$ and $x_k, \dots, x_{k+n} \in E_\mu(M)$ (with $k \in \{1, \dots, n-1\}$). Therefore, there exist scalars $\alpha_1, \dots, \alpha_n \in \mathbb{K}$ such that :

$$x = \sum_{i=1}^{n} \alpha_i x_i \\ = \sum_{i=1}^{k} \alpha_i x_i + \sum_{i=k+1}^{n} \alpha_i x_i \in E_{\lambda}(M) + E_{\mu}(M)$$

Hence the result.

Assume that $\mathbb{K}^n = E_\lambda(M) \oplus E_\mu(M)$. Let $(u_1, \dots, u_k)$ be a basis of $E_\lambda(M)$ and $(u_{k+1}, \dots, u_n)$ be a basis of $E_\mu(M)$. Then, $(u_1, \dots, u_n)$ is a basis of eigenvectors associated with the direct sum. Thus, $M$ is diagonalizable. $\square$

### Proposition 8.1.4

We have equivalence between :

(i) $M$ is diagonalizable in $\mathbb{K}$;

(ii) $\pi_M$ splits into simple roots i.e. :

$$\pi_M = \prod_{\lambda \in \text{Sp}_{\mathbb{K}}(M)} (X - \lambda).$$

**Proof.** • (i) $\implies$ (ii). Suppose that $M$ is diagonalizable in $\mathbb{K}$. It suffices to show that :

(1) $\prod_{\lambda \in \text{Sp}_{\mathbb{K}}(M)} (X - \lambda)$ divides $\pi_M$;

(2) $\pi_M$ divides $\prod_{\lambda \in \text{Sp}_{\mathbb{K}}(M)} (X - \lambda)$ by proving a simple case of the kernel lemma.

We will then deduce that since both polynomials are monic, they are therefore equal. Let us set :

$$\sum_{k=0}^{p} a_k M^k := \pi_M(X) \in \mathbb{K}[X]$$

with $p \leq n$.

(1) Let us show that $\prod_{\lambda \in \text{Sp}_{\mathbb{K}}(M)} (X - \lambda)$ divides $\pi_M$. Since the polynomials $(X - \lambda)$ for $\lambda \in \text{Sp}_{\mathbb{K}}(M)$ form a set of pairwise coprime polynomials, it suffices to show that for all $\lambda \in \text{Sp}_{\mathbb{K}}(M)$, the polynomial $(X - \lambda)$ divides $\pi_M$ i.e. that $\pi_M(\lambda) = 0$. Let $\lambda \in \text{Sp}_{\mathbb{K}}(M)$ and $x \in \mathbb{K}^n$ be an eigenvector (non-zero) associated with $\lambda$. Let us show that $\lambda$ is a root of $\pi_M$. Since $Mx = \lambda x$, we have by simple recurrence for all $k \in \mathbb{N}$ :

$$M^k x = \lambda^k x.$$

Since $\pi_M(M) = 0$, we have :

$$\begin{aligned} 0_{\mathbb{K}^n} &= 0_{\mathcal{M}_n(\mathbb{K})}x \\ &= \pi_M(M)x \\ &= \sum_{k=0}^{p} a_k M^k x \end{aligned}$$
$$ 
\begin{aligned}
&= \sum_{k=0}^{p} a_k \lambda^k x \\
&= \pi_M(\lambda)x.
\end{aligned}
$$

Since $x$ is non-zero, we have :

$$ \pi_M(\lambda) = 0. $$

(2) Let us show that $\pi_M$ divides $P := \prod_{\lambda \in \text{Sp}_K(M)} (X - \lambda)$ i.e. $P(M) = 0$ (since $P \in \text{Ann}(M)$ and by minimality of $\pi_M$, $\pi_M$ divides $P$). Let us show that :

$$ \text{Ker}(P(M)) = \bigoplus_{\lambda \in \text{Sp}_K(M)} \text{Ker}(M - \lambda I_n). \qquad (8.1.1) $$

This is a special case of the **kernel lemma**. Let us prove this result in the case where $M$ has two distinct eigenvalues $\lambda$ and $\mu$ in $\mathbb{K}$. The polynomials $X - \lambda$ and $X - \mu$ are coprime i.e. by Bézout's theorem, there exist polynomials $U, V \in \mathbb{K}[X]$ such that :

$$ (*) \quad : \quad U(X)(X - \lambda) + V(X)(X - \mu) = 1 $$

by taking for example (though it is not necessary for what follows) :

$$ U := \frac{1}{\mu - \lambda}, \quad V := \frac{1}{\lambda - \mu}. $$

Therefore, by evaluating $(*)$ at $M$, we have :

$$ (**) \quad : \quad U(M)(M - \lambda I_n) + V(M)(M - \mu I_n) = I_n. $$

Let us show that :

$$ \text{Ker}(M - \lambda I_n) \cap \text{Ker}(M - \mu I_n) = \{0\}. $$

Let $x \in \text{Ker}(M - \lambda I_n) \cap \text{Ker}(M - \mu I_n)$. According to relation (**), we have :

$$ 
\begin{aligned}
x &= I_n x \\
&= U(M)(M - \lambda I_n)x + V(M)(M - \mu I_n)x \\
&= U(M)0_{\mathbb{K}^n} + V(M)0_{\mathbb{K}^n} \\
&= 0_{\mathbb{K}^n}
\end{aligned}
$$

Let us show by double inclusion that :

$$ \text{Ker}(P(M)) = \text{Ker}(M - \lambda I_n) + \text{Ker}(M - \mu I_n). $$

Let $x \in \text{Ker}(P(M))$. Set :

$$ u := U(M)(M - \lambda I_n)x, \quad v := V(M)(M - \mu I_n)x. $$

By (**), we have :

$$ 
\begin{aligned}
(M - \mu I_n)u &= (M - \mu I_n)U(M)(M - \lambda I_n)x \\
&= U(M)P(M)x \\
&= 0 \\
(M - \lambda I_n)v &= (M - \lambda I_n)V(M)(M - \mu I_n)x \\
&= V(M)P(M)x \\
&= 0
\end{aligned}
$$
i.e. we have :

$$u \in \mathrm{Ker}(M - \mu I_n), \quad v \in \mathrm{Ker}(M - \lambda I_n).$$

Moreover, by (**), we have :

$$\begin{aligned}
x &= I_n x \\
&= U(M)(M - \lambda I_n)x + V(M)(M - \mu I_n)x \\
&= u + v \in \mathrm{Ker}(M - \mu I_n) + \mathrm{Ker}(M - \lambda I_n)
\end{aligned}$$

Let $y \in \mathrm{Ker}(M - \lambda I_n)$ i.e. $(M - \lambda I_n)y = 0$. Thus we have :

$$\begin{aligned}
P(M)y &= (M - \lambda I_n)(M - \mu I_n)y \\
&= (M - \mu I_n)(M - \lambda I_n)y \\
&= (M - \mu I_n)0 \\
&= 0
\end{aligned}$$

i.e. we have $y \in \mathrm{Ker}(P(M))$. Similarly, by symmetry, $\mathrm{Ker}(M - \mu I_n) \subset \mathrm{Ker}(P(M))$. Thus we have :

$$\mathrm{Ker}(M - \lambda I_n) + \mathrm{Ker}(M - \mu I_n) \subset \mathrm{Ker}(P(M)).$$

Thus, by lemma 8.1.3, we have :

$$\mathrm{Ker}(P(M)) = \bigoplus_{\lambda \in \mathrm{Sp}_{\mathbb{K}}(M)} \mathrm{Ker}(M - \lambda I_n) = \bigoplus_{\lambda \in \mathrm{Sp}_{\mathbb{K}}(M)} E_{\lambda}(M) = \mathbb{K}^n$$

i.e. $P(M) = 0$.

• (ii) $\implies$ (i). Conversely, suppose that $\pi_M$ splits into simple roots i.e.

$$\pi_M = \prod_{\lambda \in \mathrm{Sp}_{\mathbb{K}}(M)} (X - \lambda).$$

Then, by equation (8.1.1) :

$$\begin{aligned}
\mathbb{K}^n &= \mathrm{Ker}(0_{\mathcal{M}_n(\mathbb{K})}) \\
&= \mathrm{Ker}(\pi_M(M)) \\
&= \bigoplus_{\lambda \in \mathrm{Sp}(M)} \mathrm{Ker}(M - \lambda I_n) \\
&= \bigoplus_{\lambda \in \mathrm{Sp}(M)} E_{\lambda}(M)
\end{aligned}$$

Therefore, we can choose a basis of eigenvectors of $M$ in $\mathbb{K}^n$ adapted to this direct sum i.e. $M$ is diagonalizable in $\mathbb{K}$. $\square$

## 8.2 Example

Let us calculate the inverse of :

$$A := \begin{pmatrix}
2 & -2 & 1 \\
2 & -3 & 2 \\
-1 & 2 & 0
\end{pmatrix}$$

using the minimal polynomial of $A$. For this, we need to factorize the characteristic polynomial of $A$ as opposed to section 6. We have :

$$\chi_A(X) = \det(A - XI_3)$$
$$
\begin{align*}
&= \begin{vmatrix} 2-X & -2 & 1 \\ 2 & -3-X & 2 \\ -1 & 2 & -X \end{vmatrix} \\
&= \begin{vmatrix} 2-X & -2 & 1 \\ -2+2X & 1-X & 0 \\ -1+2X-X^2 & 2-2X & 0 \end{vmatrix} & L_2 \leftarrow L_2 - 2L_1 \\
&= \begin{vmatrix} 2-X & -2 & 1 \\ -2(1-X) & 1-X & 0 \\ -(1-X)^2 & 2(1-X) & 0 \end{vmatrix} & L_3 \leftarrow L_3 + XL_1 \\
&= \begin{vmatrix} -2(1-X) & 1-X \\ -(1-X)^2 & 2(1-X) \end{vmatrix} \\
&= (1-X)^2 \begin{vmatrix} -2 & 1 \\ -(1-X) & 2 \end{vmatrix} \\
&= -(X-1)^2 (X+3)
\end{align*}
$$

Let $x, y, z \in \mathbb{R}$. We have :

$$
\begin{align*}
\begin{pmatrix} x \\ y \\ z \end{pmatrix} \in E_1(A) &\iff A \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} x \\ y \\ z \end{pmatrix} \\
&\iff \left\{ \begin{array}{l} 2x - 2y + z = x \\ 2x - 3y + 2z = y \\ -x + 2y = z \end{array} \right. \\
&\iff \{ x - 2y + z = 0 \}
\end{align*}
$$

and we have :

$$
\begin{align*}
\begin{pmatrix} x \\ y \\ z \end{pmatrix} \in E_{-3}(A) &\iff A \begin{pmatrix} x \\ y \\ z \end{pmatrix} = -3 \begin{pmatrix} x \\ y \\ z \end{pmatrix} \\
&\iff \left\{ \begin{array}{l} 2x - 2y + z = -3x \\ 2x - 3y + 2z = -3y \\ -x + 2y = -3z \end{array} \right. \\
&\iff \left\{ \begin{array}{l} 5x - 2y + z = 0 \\ 2x + 2z = 0 \\ -x + 2y + 3z = 0 \end{array} \right. \\
&\iff \left\{ \begin{array}{l} 8y + 16z = 0 \qquad L_5 \longrightarrow L_5 + 5L_3 \\ 4y + 8z = 0 \qquad L_2 \longrightarrow L_2 + 2L_3 \\ -x + 2y + 3z = 0 \end{array} \right. \\
&\iff \left\{ \begin{array}{l} y = -2z \\ x = 2y + 3z = -z \end{array} \right.
\end{align*}
$$

Therefore, we have :

$$
E_1(A) = \mathrm{Vect} \left( \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} \right) , \quad E_{-3}(A) = \mathrm{Vect} \left( \begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix} \right).
$$

Thus, the dimensions of the eigenspaces are equal to the multiplicity orders of the eigenvalues in the characteristic polynomial *i.e.* *A* is diagonalizable in *R*. We have two possible methods.
(1) **Using the minimal annihilator polynomial.** This is what we see in this section.

(2) **Using the transition matrix** (this method is impractical as it requires calculating the inverse of the transition matrix, which may be as difficult as calculating the inverse of $A$). This is what we will see in the next section 9.

Since $A$ is diagonalizable in $\mathbb{R}$, its annihilator polynomial $\pi_A$ is split with simple real roots, is divisible by $(X-1)(X+3)$, and divides $\chi_A$ i.e. we have :

$$ \pi_A(X) = (X-1)(X+3) = X^2 + 2X - 3. $$

We verify that (but this is unnecessary) :

$$ \begin{aligned} \pi_A(A) &= A^2 + 2A - 3I_3 \\ &= \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} + 2 \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} - 3 \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \\ &= \begin{pmatrix} -1 & 4 & -2 \\ -4 & 9 & -4 \\ 2 & -4 & 3 \end{pmatrix} + \begin{pmatrix} -1 & 4 & -2 \\ -4 & 9 & -4 \\ 2 & -4 & 3 \end{pmatrix} \\ &= 0 \end{aligned} $$

i.e. we have $A(A + 2I_3) = 3I_3$. Therefore, we have :

$$ \begin{aligned} A^{-1} &= \frac{1}{3}(A + 2I_3) \\ &= \frac{1}{3} \left( \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} + 2 \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \right) \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{aligned} $$
# 9 Diagonalization and transition matrix

We will resume the calculations from the previous section 8 and use the transition matrix to calculate the inverse of an invertible and diagonalizable matrix.

## 9.1 Introduction

There is a simple relationship between the inverse of the matrix, the transition matrix, and the diagonal matrix.

**Proposition 9.1.1**

Let $A \in \text{GL}_n(\mathbb{R})$. Suppose that $A$ is diagonalizable in $\mathbb{R}$ i.e. there exists a matrix $P \in \text{GL}_n(\mathbb{R})$ and a diagonal matrix :

$$D := \begin{pmatrix} \lambda_1 & 0 & \cdots & 0 \\ 0 & \lambda_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & \cdots & 0 & \lambda_n \end{pmatrix}.$$

such that :

$$A = PDP^{-1}$$

Then we have :

$$A^{-1} = PD^{-1}P^{-1} = P \begin{pmatrix} 1/\lambda_1 & 0 & \cdots & 0 \\ 0 & 1/\lambda_2 & \cdots & 0 \\ \vdots & \vdots & \ddots & \vdots \\ 0 & \cdots & 0 & 1/\lambda_n \end{pmatrix} P^{-1}.$$

**Proof.** We have :

$$\begin{pmatrix} \lambda_1 & & (0) \\ & \ddots & \\ (0) & & \lambda_n \end{pmatrix} \begin{pmatrix} 1/\lambda_1 & & (0) \\ & \ddots & \\ (0) & & 1/\lambda_n \end{pmatrix} = \begin{pmatrix} 1 & & (0) \\ & \ddots & \\ (0) & & 1 \end{pmatrix}.$$

therefore :

$$D^{-1} = \begin{pmatrix} 1/\lambda_1 & & (0) \\ & \ddots & \\ (0) & & 1/\lambda_n \end{pmatrix}.$$

Moreover, we have :

$$\begin{align*} PD^{-1}P^{-1}A &= PD^{-1}P^{-1}PDP^{-1} \\ &= I_n \end{align*}$$

Hence the result.

<figure><img src="image_5.png" /></figure>

This method is not practical because it requires calculating the inverse of $P$, which is not necessarily easier than calculating the inverse of $A$.

## 9.2 Example

Let's calculate the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$
using the transition matrix. We found in the previous section 8 that the characteristic polynomial of A is given by :

$$ \chi_A(X) = -(X-1)^2 (X+3) $$

and that :

$$ E_1(A) = \mathrm{Vect}\left(\begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix}\right), \quad E_{-3}(A) = \mathrm{Vect}\left(\begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix}\right). $$

So A is diagonalizable. Let us set :

$$ P := \begin{pmatrix} 2 & -1 & -1 \\ 1 & 0 & -2 \\ 0 & 1 & 1 \end{pmatrix}, \quad D := \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & -3 \end{pmatrix}. $$

We have, for example, using the adjugate matrix (see Corollary 1.1.3) :

$$ P^{-1} = \frac{1}{\det(P)} \mathrm{Com}(P)^T. $$

• Adjugate of P. We have :

$$ \begin{aligned} \mathrm{Com}(P) &= \begin{pmatrix} \Delta_{11}(P) & -\Delta_{12}(P) & \Delta_{13}(P) \\ -\Delta_{21}(P) & \Delta_{22}(P) & -\Delta_{23}(P) \\ \Delta_{31}(P) & -\Delta_{32}(P) & \Delta_{33}(P) \end{pmatrix} \\ &= \begin{pmatrix} \begin{vmatrix} 0 & -2 \\ 1 & 1 \end{vmatrix} & - \begin{vmatrix} 1 & -2 \\ 0 & 1 \end{vmatrix} & \begin{vmatrix} 1 & 0 \\ 0 & 1 \end{vmatrix} \\ - \begin{vmatrix} -1 & -1 \\ 1 & 1 \end{vmatrix} & \begin{vmatrix} 2 & -1 \\ 0 & 1 \end{vmatrix} & - \begin{vmatrix} 2 & -1 \\ 0 & 1 \end{vmatrix} \\ \begin{vmatrix} -1 & -1 \\ 0 & -2 \end{vmatrix} & - \begin{vmatrix} 2 & -1 \\ 1 & -2 \end{vmatrix} & \begin{vmatrix} 2 & -1 \\ 1 & 0 \end{vmatrix} \end{pmatrix} \\ &= \begin{pmatrix} 2 & -1 & 1 \\ 0 & 2 & -2 \\ 2 & 3 & 1 \end{pmatrix} \end{aligned} $$

• Determinant of P. We have :

$$ \begin{aligned} \det(P) &= \begin{vmatrix} 2 & -1 & -1 \\ 1 & 0 & -2 \\ 0 & 1 & 1 \end{vmatrix} \\ &= 2 \begin{vmatrix} 0 & -2 \\ 1 & 1 \end{vmatrix} - \begin{vmatrix} -1 & -1 \\ 1 & 1 \end{vmatrix} \\ &= 4 \end{aligned} $$

Thus we have :

$$ \begin{aligned} P^{-1} &= \frac{1}{\det(P)} \mathrm{Com}(P)^T \\ &= \frac{1}{4} \begin{pmatrix} 2 & -1 & 1 \\ 0 & 2 & -2 \\ 2 & 3 & 1 \end{pmatrix}^T \end{aligned} $$
$$ = \frac{1}{4} \begin{pmatrix} 2 & 0 & 2 \\ -1 & 2 & 3 \\ 1 & -2 & 1 \end{pmatrix} $$

* Calculation of $A^{-1}$. Since $D = P^{-1}AP$, we have

$$ \begin{aligned} A^{-1} &= PD^{-1}P^{-1} \\ &= \frac{1}{4} \begin{pmatrix} 2 & -1 & -1 \\ 1 & 0 & -2 \\ 0 & 1 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & -1/3 \end{pmatrix} \begin{pmatrix} 2 & 0 & 2 \\ -1 & 2 & 3 \\ 1 & -2 & 1 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 6 & -3 & 1 \\ 3 & 0 & 2 \\ 0 & 3 & -1 \end{pmatrix} \begin{pmatrix} 2 & 0 & 2 \\ -1 & 2 & 3 \\ 1 & -2 & 1 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{aligned} $$
# 10 Diagonalization and Orthogonal Matrix

The issue from the previous section 9 is that it requires calculating the inverse of the transition matrix. There is a simple case where the inverse of the transition matrix is straightforward to compute : if the starting matrix is symmetric, then the transition matrix is orthogonal. What if the starting matrix is not symmetric ? In that case, we can work with an associated matrix, which is the product of the starting matrix and its transpose.

We might have hoped that the method would be simpler, but as we will see, due to orthonormalization, square roots will appear throughout the calculations, making the method ultimately lengthy and technical.

## 10.1 Introduction

### Definition 10.1.1

Let $A, B \in \mathcal{M}_n(\mathbb{R})$.

(i) We say that $A$ and $B$ are **orthogonally similar** if there exists a matrix $P \in O_n(\mathbb{R})$ (i.e. $P^{-1} = P^T$) such that :

$$A = P^T B P.$$

We then denote :

$$A \sim_{\perp} B.$$

(ii) We say that $A$ is **orthogonally diagonalizable** if there exists a diagonal matrix $D \in \mathcal{M}_n(\mathbb{R})$ such that :

$$A \sim_{\perp} D.$$

We start with a usual lemma about the eigenvalues and eigenvectors of a symmetric matrix.

### Lemma 10.1.2

Let $C \in \mathcal{M}_n(\mathbb{R})$ be **symmetric** i.e. $C^T = C$.

(i) Then $\text{Sp}_C(C) \subset \mathbb{R}$.

(ii) Suppose there are two distinct eigenvalues $\lambda$ and $\mu$ of $C$. Let $x \in E_\lambda(C)$ and $y \in E_\mu(C)$. Then we have :

$$\langle x, y \rangle = 0.$$

### Proof.

(i) Let $\lambda \in \text{Sp}_C(C)$ and $x \in \mathbb{C}^n$ be an eigenvector associated with $\lambda$ (non-zero). Since $x$ is non-zero, we have :

$$\alpha := \|x\|^2 = \bar{x}^T x > 0$$

Thus we have :

$$\begin{align*}
\lambda &= \frac{\lambda \bar{x}^T x}{\alpha} \\
&= \frac{\bar{x}^T (\lambda x)}{\alpha} \\
&= \frac{\bar{x}^T C x}{\alpha} \\
&= \frac{Cx^T x}{\alpha}
\end{align*}$$
$$ 
\begin{aligned}
&= \frac{\bar{\lambda} x^T x}{\alpha} \\
&= \frac{\bar{\lambda} \bar{x}^T x}{\alpha} \\
&= \bar{\lambda}
\end{aligned}
$$

i.e. $\lambda \in \mathbb{R}$.

(ii) We have :

$$ 
\begin{aligned}
(\lambda - \mu)\langle x, y \rangle &= \lambda x^T y - \mu x^T y \\
&= (\lambda x)^T y - x^T (\mu y) \\
&= (Cx)^T y - x^T (Cy) \\
&= x^T C^T y - x^T Cy \\
&= x^T Cy - x^T Cy \\
&= 0
\end{aligned}
$$

where $\lambda - \mu \neq 0$, hence the result.

<figure><img src="image_6.png" /></figure>

We then deduce the spectral theorem.

### Proposition 10.1.3: Spectral Theorem

Let $C \in \mathcal{M}_n(\mathbb{R})$ be symmetric. Then $C$ is orthogonally diagonalizable.

**Proof.** A usual proof of the spectral theorem is done by induction on the dimension.

We consider here the dimension $n := 3$ and construct the orthogonal matrix by hand. The matrix $C$ has 3 real eigenvalues (counted with multiplicity) according to lemma 10.1.2.

Let $\lambda \in \text{Sp}(C)$. There exists $u \in \mathbb{R}^3 - \{0\}$ such that $Cu = \lambda u$. Let us define :

$$ F := \text{Vect}(u)^\perp = \{x \in \mathbb{R}^3, \langle u, x \rangle = 0\} $$

the orthogonal complement of $\text{Vect}(u)$ in $\mathbb{R}^3$. Then $F$ is a vector subspace of dimension 2 (the orthogonal of a line in dimension 3 is a vector plane) that is invariant under $C$ because for all $y \in F$ :

$$ 
\begin{aligned}
\langle Cy, u \rangle &= (Cy)^T u \\
&= y^T C^T u \\
&= y^T (Cu) \\
&= \lambda y^T u \\
&= \lambda \langle y, u \rangle \\
&= 0.
\end{aligned}
$$

i.e. $Cy \in F$.

Let $(v, w)$ be an orthonormal basis of $F$. Let $\mathcal{B} := (u, v, w)$ be the orthonormal basis and $P := (u | v | w)$ be the orthogonal matrix. Since $F$ is invariant under $C$, there exist $a, b, c, d \in \mathbb{R}$ such that $Cv = av + bw$ and $Cw = cv + dw$. We then have, by change of basis :

$$ C' := P^T C P $$
$$ 
\begin{aligned}
&= (\mathrm{Mat}_{\mathcal{B}}(Cu) | \mathrm{Mat}_{\mathcal{B}}(Cv) | \mathrm{Mat}_{\mathcal{B}}(Cw)) \\
&= (\mathrm{Mat}_{\mathcal{B}}(\lambda u) | \mathrm{Mat}_{\mathcal{B}}(av + bw) | \mathrm{Mat}_{\mathcal{B}}(cv + dw)) \\
&:= \begin{pmatrix} \lambda & 0 \\ 0 & \widetilde{C}' \end{pmatrix}
\end{aligned}
$$

where :

$$ \widetilde{C}' := \begin{pmatrix} a & c \\ b & d \end{pmatrix}. $$

The matrix $C'$ is symmetric because :

$$ 
\begin{aligned}
(C')^T &= (P^T C P)^T \\
&= P^T C^T P \\
&= P^T C P \\
&= C'
\end{aligned}
$$

Thus $b = c$ i.e. $\widetilde{C}'$ is symmetric.

Using lemma 10.1.2 again, let $\mu$ be a real eigenvalue of $\widetilde{C}'$. There exists $u' \in \mathbb{R}^2 - \{0\}$ such that $\widetilde{C}'u' = \mu u'$. Let us define :

$$ F' := \mathrm{Vect}(u')^{\perp} = \{x \in \mathbb{R}^2, \langle u', x \rangle = 0\} $$

the orthogonal complement of $\mathrm{Vect}(u')$ in $\mathbb{R}^2$. Then $F'$ is a vector subspace of dimension 1 (the orthogonal of a line in dimension 2 is a line) that is invariant under $\widetilde{C}'$ because for all $y \in F'$ :

$$ 
\begin{aligned}
\langle \widetilde{C}'y, u' \rangle &= (\widetilde{C}'y)^T u' \\
&= y^T \widetilde{C}'^T u' \\
&= y^T (\widetilde{C}'u') \\
&= \mu y^T u' \\
&= \mu \langle y, u' \rangle \\
&= 0.
\end{aligned}
$$

i.e. $\widetilde{C}'y \in F'$.

Let $v'$ be an orthonormal basis of $F'$. Let $\mathcal{B}' := (u', v')$ be the orthonormal basis and $\widetilde{P}' := (u' | v')$ be the orthogonal matrix. Since $F'$ is invariant under $\widetilde{C}'$, there exists $\nu \in \mathbb{R}$ such that $\widetilde{C}'v' = \nu v'$. We have :

$$ 
\begin{aligned}
\widetilde{C}'' &:= (\widetilde{P}')^T \widetilde{C}' \widetilde{P}' \\
&= (\mathrm{Mat}_{\mathcal{B}'}(\widetilde{C}'u') | \mathrm{Mat}_{\mathcal{B}'}(\widetilde{C}'v')) \\
&= (\mathrm{Mat}_{\mathcal{B}'}(\mu u') | \mathrm{Mat}_{\mathcal{B}'}(\nu v')) \\
&= \begin{pmatrix} \mu & 0 \\ 0 & \nu \end{pmatrix}
\end{aligned}
$$

Let us define :

$$ P' := \begin{pmatrix} 1 & 0 \\ 0 & \widetilde{P}' \end{pmatrix} $$

Then the matrix $P'$ is orthogonal because :

$$ (P')^T P' = \begin{pmatrix} 1 & 0 \\ 0 & \widetilde{P}' \end{pmatrix}^T \begin{pmatrix} 1 & 0 \\ 0 & \widetilde{P}' \end{pmatrix} $$
$$ 
\begin{aligned}
&= \begin{pmatrix} 1 & 0 \\ 0 & \widetilde{P'}^T \widetilde{P'} \end{pmatrix} \\
&= I_3
\end{aligned}
$$

and thus also $PP'$. And we have :

$$ 
\begin{aligned}
(PP')^T CPP' &= (P')^T P^T CPP' \\
&= \begin{pmatrix} 1 & 0 \\ 0 & \widetilde{P'} \end{pmatrix}^T \begin{pmatrix} \lambda & 0 \\ 0 & \widetilde{C'} \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \widetilde{P'} \end{pmatrix} \\
&= \begin{pmatrix} \lambda & 0 \\ 0 & (\widetilde{P'})^T \widetilde{C'} \widetilde{P'} \end{pmatrix} \\
&= \begin{pmatrix} \lambda & 0 & 0 \\ 0 & \mu & 0 \\ 0 & 0 & \nu \end{pmatrix}
\end{aligned}
$$

i.e. we have :

$$ C \sim_{\perp} \begin{pmatrix} \lambda & 0 & 0 \\ 0 & \mu & 0 \\ 0 & 0 & \nu \end{pmatrix}. $$

We then deduce a method to compute the inverse of a matrix without computing the inverse of the transition matrix. Let $A \in GL_n(\mathbb{R})$. Define :

$$ C := AA^T. $$

Then $C$ is symmetric because :

$$ 
\begin{aligned}
C^T &= (AA^T)^T \\
&= AA^T \\
&= C.
\end{aligned}
$$

Thus there exists an orthogonal matrix $P$ and a diagonal matrix $D \in \mathcal{M}_n(\mathbb{R})$ such that :

$$ C = PDP^{-1} = PDP^T. $$

Since $C^{-1} = (A^T)^{-1}A^{-1}$, we then have :

$$ 
\begin{aligned}
A^{-1} &= A^T C^{-1} \\
&= A P D^{-1} P^T C^{-1}
\end{aligned}
$$

So we only need to "simply" compute the inverse of a diagonal matrix. As mentioned in the introduction, this will not prevent the computation from being tedious since calculating $P$ will involve many square roots.

## 10.2 Example

Let's calculate the inverse of :

$$ A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} $$

by diagonalizing the symmetric matrix :

$$ C := AA^T. $$
We have :

$$
\begin{align*}
C &= AA^T \\
  &= \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 2 & 2 & -1 \\ -2 & -3 & 2 \\ 1 & 2 & 0 \end{pmatrix} \\
  &= \begin{pmatrix} 9 & 12 & -6 \\ 12 & 17 & -8 \\ -6 & -8 & 5 \end{pmatrix}
\end{align*}
$$

The matrix $C$ is diagonalizable in $\mathbb{R}$ because it is symmetric. We have :

$$
\begin{align*}
\chi_C(X) &= \det (C - XI_3) \\
&= \begin{vmatrix}
9-X & 12 & -6 \\
12 & 17-X & -8 \\
-6 & -8 & 5-X
\end{vmatrix} \\
&= \begin{vmatrix}
0 & 8X & -36 + (9-X)(5-X) \\
0 & 1-X & 2-2X \\
6 & -8 & 5-X
\end{vmatrix} & L_1 \leftarrow 6L_1 + (9-X)L_3 \\
&= 6 \begin{vmatrix}
8X & -36 + (9-X)(5-X) \\
1-X & 2-2X
\end{vmatrix} & L_2 \leftarrow L_2 + 2L_3 \\
&= 6(1-X) \begin{vmatrix}
8X & 9-14X+X^2 \\
1 & 2
\end{vmatrix} \\
&= 6(1-X)(X^2-30X+9)
\end{align*}
$$

Let $e \in \{\pm 1\}$ :

$a := 6\sqrt{6} \ , \ \lambda_e := 15 + ea.$

Thus :

$$
\mathrm{Sp}(C) = \{1, 15 \pm 6\sqrt{6}\} = \{1, \lambda_1, \lambda_{-1}\}.
$$

Let $x, y, z \in \mathbb{R}$. We have :

$$
\begin{align*}
\begin{pmatrix} x \\ y \\ z \end{pmatrix} \in E_1(C) &\iff C \begin{pmatrix} x \\ y \\ z \end{pmatrix} = \begin{pmatrix} x \\ y \\ z \end{pmatrix} \\
&\iff \begin{cases} 9x + 12y - 6z = x \\ 12x + 17y - 8z = y \\ -6x - 8y + 5z = z \end{cases} \\
&\iff \begin{cases} 4x + 6y - 3z = 0 \\ 6x + 8y - 4z = 0 \\ -3x - 4y + 2z = 0 \end{cases} \quad L_2 = -2L_3 \\
&\iff \begin{cases} 4x + 6y - 3z = 0 \\ -3x - 4y + 2z = 0 \end{cases} \\
&\iff \begin{cases} 4x + 6y - 3z = 0 \\ 2y - z = 0 \end{cases} \quad L_2 \leftarrow 4L_2 + 3L_1 \\
&\iff \begin{cases} x = 0 \\ z = 2y \end{cases}
\end{align*}
$$
and for $e \in \{\pm 1\}$:

$$
\begin{align*}
\begin{pmatrix} x \\ y \\ z \end{pmatrix} \in E_{\lambda_e}(C) &\iff C \begin{pmatrix} x \\ y \\ z \end{pmatrix} = (15 + ea) \begin{pmatrix} x \\ y \\ z \end{pmatrix} \\
&\iff \left\{
\begin{array}{l}
9x + 12y - 6z = (15 + ea)x \\
12x + 17y - 8z = (15 + ea)y \\
-6x - 8y + 5z = (15 + ea)z
\end{array}
\right. \\
&\iff \left\{
\begin{array}{l}
-(6 + ea)x + 12y - 6z = 0 \\
12x + (2 - ea)y - 8z = 0 \\
-6x - 8y - (10 + ea)z = 0
\end{array}
\right. \\
&\iff \left\{
\begin{array}{l}
(120 + 8ea)y + 2(120 + 8ea)z = 0 \\
-(14 + ea)y - 2(14 + ea)z = 0 \\
6x - 8y - (10 + ea)z = 0
\end{array}
\right. \qquad \begin{array}{l} L_1 \leftarrow 6L_1 - (6 + ea)L_3 \\ L_2 \leftarrow L_2 + 2L_3 \end{array} \\
&\iff \left\{
\begin{array}{l}
y = -2z \\
x = \frac{8y + (10 + ea)}{6} \quad z = \frac{-6 + ea}{6} \quad z = (1 - e\sqrt{6})z
\end{array}
\right.
\end{align*}
$$

Thus, by normalizing each vector :

$$
E_1(C) = \operatorname{Vect} \begin{pmatrix} 0 \\ 1 \\ 2 \end{pmatrix} = \operatorname{Vect} \begin{pmatrix} \frac{1}{\sqrt{5}} & 0 \\ 1 & 2 \end{pmatrix}
$$

$$
E_{\lambda_e}(C) = \operatorname{Vect} \begin{pmatrix} 1 - e\sqrt{6} \\ -2 \\ 1 \end{pmatrix} = \operatorname{Vect} \left( \alpha_e \begin{pmatrix} 1 - e\sqrt{6} \\ -2 \\ 1 \end{pmatrix} \right)
$$

where for $e \in \{\pm 1\}$:

$$
\alpha_e := \frac{1}{\sqrt{12 - 2e\sqrt{6}}}
$$

Thus the orthogonal change of basis matrix is :

$$
P := \begin{pmatrix} 0 & \alpha_1 (1 - \sqrt{6}) & \alpha_{-1} (1 + \sqrt{6}) \\ 1/\sqrt{5} & -2\alpha_1 & -2\alpha_{-1} \\ 2/\sqrt{5} & \alpha_1 & \alpha_{-1} \end{pmatrix}.
$$

Let:

$$
D := \begin{pmatrix} 1 & 0 & 0 \\ 0 & \lambda_1 & 0 \\ 0 & 0 & \lambda_{-1} \end{pmatrix} := \begin{pmatrix} 1 & 0 & 0 \\ 0 & 15 + 6\sqrt{6} & 0 \\ 0 & 0 & 15 - 6\sqrt{6} \end{pmatrix}.
$$

Thus :

$$
\begin{align*}
C^{-1} &= PD^{-1}P^T \\
&= \begin{pmatrix} 0 & \alpha_1(1-\sqrt{6}) & \alpha_{-1}(1+\sqrt{6}) \\ 1/\sqrt{5} & -2\alpha_1 & -2\alpha_{-1} \\ 2/\sqrt{5} & \alpha_1 & \alpha_{-1} \end{pmatrix}
\begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/\lambda_1 & 0 \\ 0 & 0 & 1/\lambda_{-1} \end{pmatrix}
\begin{pmatrix} 0 & 1/\sqrt{5} & 2/\sqrt{5} \\ \alpha_1(1-\sqrt{6}) & -2\alpha_1 & \alpha_1 \\ \alpha_{-1}(1+\sqrt{6}) & -2\alpha_{-1} & \alpha_{-1} \end{pmatrix} \\
&= \begin{pmatrix} 0 & \alpha_1(1-\sqrt{6}) & \alpha_{-1}(1+\sqrt{6}) \\ 1/\sqrt{5} & -2\alpha_1 & -2\alpha_{-1} \\ 2/\sqrt{5} & \alpha_1 & \alpha_{-1} \end{pmatrix}
\begin{pmatrix} 0 & 1/\sqrt{5} & 2/\sqrt{5} \\ \alpha_1(1-\sqrt{6})/\lambda_1 & -2\alpha_1/\lambda_1 & \alpha_1/\lambda_1 \\ \alpha_{-1}(1+\sqrt{6})/\lambda_{-1} & -2\alpha_{-1}/\lambda_{-1} & \alpha_{-1}/\lambda_{-1} \end{pmatrix}
\end{align*}
$$
$$ = \frac{1}{3} \begin{pmatrix} 7 & -4 & 2 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} $$

Thus, since $C^{-1} = (A^T)^{-1}A^{-1}$, we have :

$$ \begin{align*} A^{-1} &= A^T C^{-1} \\ &= \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ -2 & -3 & 2 \\ 1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 7 & -4 & 2 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{align*} $$
# 11 Schur Decomposition

The Schur decomposition allows us to triangularize a triangularizable matrix in $\mathbb{R}$ with an orthogonal change of basis matrix. We obtain a simple algorithm : the basic ingredient is the construction of successive orthonormal bases with the first vector being the normalized eigenvector.

## 11.1 Introduction

Recall the definition of a triangularizable matrix.

**Definition 11.1.1**

(i) We say that $M$ is **triangularizable (in $\mathbb{R}$)** if $M$ is similar to a triangular matrix, i.e. there exists $P \in GL_n(\mathbb{R})$ and a triangular matrix $T \in \mathcal{M}_n(\mathbb{R})$ such that :

$$M = PTP^{-1}.$$

(ii) We say that $M$ is **orthogonally triangularizable (in $\mathbb{R}$)** if $M$ is orthogonally similar to a triangular matrix, i.e. there exists $P \in O_n(\mathbb{R})$ and a triangular matrix $T \in \mathcal{M}_n(\mathbb{R})$ such that :

$$M = PTP^T.$$

Let :

$$T := \begin{pmatrix} \lambda_1 & & (0) \\ & \ddots & \\ (0) & & \lambda_n \end{pmatrix}.$$

Since :

$$\begin{align*} \chi_M(X) &= \det(M - XI_n) \\ &= \det(PTP^{-1} - XI_n) \\ &= \det(T - XI_n) \\ &= \chi_T(X) \\ &= \prod_{i=1}^{n} (\lambda_i - X) \end{align*}$$

it follows that $M$ has $n$ real eigenvalues (counted with multiplicity).

We start with a lemma.

**Lemma 11.1.2**

Let $M \in \mathcal{M}_n(\mathbb{R})$ be a triangularizable matrix in $\mathbb{R}$, and $(u_1, \dots, u_n)$ a basis of eigenvectors associated respectively with the real eigenvalues $\lambda_1, \dots, \lambda_n$ of $M$. Let :

$$w_1 := \frac{u_1}{\|u_1\|}.$$

Complete $w_1$ into an orthonormal basis $(w_1, \dots, w_n)$ of $\mathbb{R}^n$. Define the matrix :

$$Q := (w_1 | \cdots | w_n) \in O_n(\mathbb{R}).$$
Then $Q^T M Q$ is of the type :

$$ \begin{pmatrix} \lambda_1 & * \\ 0 & M' \end{pmatrix} := Q^T M Q $$

with $M' \in \mathcal{M}_{n-1}(\mathbb{R})$. The matrix $M'$ also satisfies the following two properties.

(i) We have :

$$ \mathrm{Sp}(M') = \{\lambda_2, \dots, \lambda_n\}. $$

(ii) For each $i \in \{2, \dots, n\}$, let $u'_i \in \mathbb{R}^{n-1}$ such that :

$$ Q^T u_i = \begin{pmatrix} * \\ u'_i \end{pmatrix}. $$

Then $u'_i \in E_{\lambda_i}(M')$.

**Proof.** Through the change of basis given by the matrix $Q$, the first column of the matrix $Q^T M Q = Q^{-1} M Q$ is the decomposition of the vector $Mw_1 = \lambda_1 w_1$ in the basis $(w_1, \dots, w_n)$ i.e. this first column is simply :

$$ \begin{pmatrix} \lambda_1 \\ 0 \end{pmatrix}. $$

Hence the form of the matrix $Q^T M Q$.

(i) We have :

$$ \begin{aligned} \chi_M(X) &= \det(M - XI_n) \\ &= \det(Q^T M Q - XI_n) \\ &= \begin{vmatrix} \lambda_1 - X & * \\ 0 & M' - XI_{n-1} \end{vmatrix} \\ &= (\lambda_1 - X) \det(M' - XI_{n-1}) \\ &= (\lambda_1 - X) \chi_{M'}(X) \end{aligned} $$

Thus, the result follows.

(ii) We have :

$$ \begin{aligned} \begin{pmatrix} * \\ \lambda_i u'_i \end{pmatrix} &= \lambda_i \begin{pmatrix} * \\ u'_i \end{pmatrix} \\ &= \lambda_i Q^T u_i \\ &= Q^T M u_i \\ &= Q^T M Q Q^T u_i \\ &= \begin{pmatrix} \lambda_1 & b^T \\ 0 & M' \end{pmatrix} Q^T u_i \\ &= \begin{pmatrix} \lambda_1 & b^T \\ 0 & M' \end{pmatrix} \begin{pmatrix} * \\ u'_i \end{pmatrix} \\ &= \begin{pmatrix} * \\ M' u'_i \end{pmatrix} \end{aligned} $$

Therefore, we have $M'u'_i = \lambda_i u'_i$.
<figure><img src="image_7.png" /></figure>

We thus have a simple algorithm to obtain the Schur decomposition; it is enough to reapply the lemma with the matrix $M'$. We then obtain the following result.

**Proposition 11.1.3: Schur Decomposition**

Let $M \in \mathcal{M}_n(\mathbb{R})$ be a triangularizable matrix in $\mathbb{R}$. Then $M$ is orthogonally triangularizable i.e. there exists an upper triangular matrix $T$ and an orthogonal matrix $Q$ such that :

$$M = QTQ^T.$$

**Proof.** Let's explain the algorithm in the case $n := 3$. Let $(u_{1,0}, u_{2,0}, u_{3,0})$ be a basis of eigenvectors associated respectively with $\lambda_1, \lambda_2, \lambda_3$ the three real eigenvalues of $M$ (counted with multiplicity).

**First step.** Let :

$$w_{1,0} := \frac{u_{1,0}}{\|u_{1,0}\|}.$$

Construct an orthonormal basis $\mathcal{B}_0 := (w_{1,0}, w_{2,0}, w_{3,0})$ with the first vector $w_{1,0}$ by the Gram-Schmidt orthonormalization process. Let :

$$Q_0 := (w_{1,0} \mid w_{2,0} \mid w_{3,0}).$$

Thus we have :

$$
\begin{aligned}
Q_0^T A Q_0 &= \begin{pmatrix} w_{1,0} \\ w_{2,0} \\ w_{3,0} \end{pmatrix} A \begin{pmatrix} w_{1,0} & w_{2,0} & w_{3,0} \end{pmatrix} \\
&= (\mathrm{Mat}_{\mathcal{B}_0}(Aw_{1,0}) | \mathrm{Mat}_{\mathcal{B}_0}(Aw_{2,0}) | \mathrm{Mat}_{\mathcal{B}_0}(Aw_{3,0})) \\
&= (\mathrm{Mat}_{\mathcal{B}_0}(\lambda_1 w_{1,0}) | \mathrm{Mat}_{\mathcal{B}_0}(Aw_{2,0}) | \mathrm{Mat}_{\mathcal{B}_0}(Aw_{3,0})) \\
&:= \begin{pmatrix} \lambda_1 & * \\ 0 & A_1 \end{pmatrix}
\end{aligned}
$$

with :

$$A_1 \in \mathrm{GL}_2(\mathbb{R}).$$

**Second step.** Let $u_{2,1}, u_{3,1} \in \mathbb{R}^2$ such that :

$$Q_0^T u_{2,0} = \begin{pmatrix} * \\ u_{2,1} \end{pmatrix}$$

$$Q_0^T u_{3,0} = \begin{pmatrix} * \\ u_{3,1} \end{pmatrix}$$

By lemma 11.1.2, we have $\mathrm{Sp}(A_1) = \{\lambda_2, \lambda_3\}$ and

$$E_{\lambda_2}(A_1) = \mathrm{Vect}(u_{2,1}), \quad E_{\lambda_3}(A_1) = \mathrm{Vect}(u_{3,1}).$$

**Third step.** Let :

$$w_{2,1} := \frac{u_{2,1}}{\|u_{2,1}\|}.$$

Construct by the Gram-Schmidt orthonormalization process (see section 13 for details on this process) an orthonormal basis $\mathcal{B}_1 := (w_{2,1}, w_{3,1})$ with the first vector $w_{2,1}$. Let :

$$\tilde{Q}_1 := (w_{2,1} \mid w_{3,1}) , \quad Q_1 := \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1 \end{pmatrix}.$$
The matrix $\tilde{Q}_1$ is orthogonal. The matrix $Q_1$ is also orthogonal because :

$$
\begin{align*}
Q_1 Q_1^T &= \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1^T \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1 \tilde{Q}_1^T \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 \\ 0 & I_2 \end{pmatrix} \\
&= I_3
\end{align*}
$$

Thus, we have :

$$
\begin{align*}
\tilde{Q}_1^T A_1 \tilde{Q}_1 &= \begin{pmatrix} w_{2,1} \\ w_{3,1} \end{pmatrix} A_1 \begin{pmatrix} w_{2,1} & \\ & w_{3,1} \end{pmatrix} \\
&= (\mathrm{Mat}_{\mathcal{B}_1}(A_1 w_{2,1}) | \mathrm{Mat}_{\mathcal{B}_1}(A_1 w_{3,1})) \\
&= (\mathrm{Mat}_{\mathcal{B}_1}(\lambda_2 w_{2,1}) | \mathrm{Mat}_{\mathcal{B}_1}(A_1 w_{3,1})) \\
&:= \begin{pmatrix} \lambda_2 & \\ 0 & \lambda_3 \end{pmatrix}^*
\end{align*}
$$

And thus, we have :

$$
\begin{align*}
T &:= Q_1^T \begin{pmatrix} \lambda_1 & \\ 0 & A_1 \end{pmatrix} Q_1 \\
&= \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1^T \end{pmatrix} \begin{pmatrix} \lambda_1 & \\ 0 & A_1 \end{pmatrix} \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1 \end{pmatrix} \\
&= \begin{pmatrix} \lambda_1 & \\ 0 & \tilde{Q}_1^T A_1 \tilde{Q}_1 \end{pmatrix} \\
&= \begin{pmatrix} \lambda_1 & \\ 0 & \lambda_2 \end{pmatrix} \begin{pmatrix} (*) \\ \lambda_3 \end{pmatrix}
\end{align*}
$$

Let :

$Q := Q_0 Q_1.$

The matrix $Q$ is orthogonal and we have :

$$
\begin{align*}
Q^T A Q &= Q_1^T Q_0^T A Q_0 Q_1 \\
&= Q_1^T \begin{pmatrix} \lambda_1 & \\ 0 & A_1 \end{pmatrix} Q_1 \\
&= T
\end{align*}
$$

Thus we have :

$$
A = Q T Q^T = Q \begin{pmatrix} \lambda_1 & & (*) \\ & \lambda_2 & \\ (0) & & \lambda_3 \end{pmatrix} Q^T
$$

i.e. $A \sim_{\perp} T$.

<figure><img src="image_8.png" /></figure>

Let $\lambda_1, \dots, \lambda_n$ be the real eigenvalues of $A$ counted with multiplicity and $u_{1,0}, \dots, u_{n,0}$ the eigenvectors associated respectively. We construct successively orthogonal matrices $Q_0, \dots, Q_{n-2}$ by recurrence on the size $n$ of the matrix. Let's explain the first two steps.
(1) Let :

$$ \mathcal{B}_0 := \left( w_{1,0} := \frac{u_{1,0}}{\|u_{1,0}\|}, w_{2,0}, \dots, w_{n,0} \right) $$

be an orthonormal basis with the first vector being the normalized $u_{1,0}$. Let :

• the orthogonal matrix :

$$ Q_0 := (w_{1,0} \mid w_{2,0} \mid \dots \mid w_{n,0}) $$

• $A_1 \in \text{GL}_{n-1}(\mathbb{R})$ the matrix such that :

$$ Q_0^T A Q_0 = \begin{pmatrix} \lambda_1 & * \\ 0 & A_1 \end{pmatrix}. $$

• let $u_{2,1}, \dots, u_{n,1} \in \mathbb{R}^{n-1}$ such that

$$ Q_0^T u_{2,0} = \begin{pmatrix} * \\ u_{2,1} \end{pmatrix}, \quad Q_0^T u_{3,0} = \begin{pmatrix} * \\ u_{3,1} \end{pmatrix} \dots Q_0^T u_{n,0} = \begin{pmatrix} * \\ u_{n,1} \end{pmatrix}. $$

By lemma 11.1.2, we have $\text{Sp}(A_1) = \{\lambda_2, \dots, \lambda_n\}$ and for all $i \in \{2, \dots, n\}$, $u_{i,1} \in E_{\lambda_i}(A_1)$.

(2) Let :

$$ \mathcal{B}_1 := \left( w_{2,1} := \frac{u_{2,1}}{\|u_{2,1}\|}, w_{3,1}, \dots, w_{n,1} \right) $$

be an orthonormal basis with the first vector being the normalized $u_{2,1}$. Let :

• the orthogonal matrices :

$$ \tilde{Q}_1 := (w_{2,1} \mid w_{3,1} \mid \dots \mid w_{n,1}), \quad Q_1 := \begin{pmatrix} 1 & 0 \\ 0 & \tilde{Q}_1 \end{pmatrix} $$

• $A_2 \in \text{GL}_{n-1}(\mathbb{R})$ the matrix such that :

$$ \tilde{Q}_1^T A_1 \tilde{Q}_1 = \begin{pmatrix} \lambda_2 & * \\ 0 & A_2 \end{pmatrix}. $$

• let $u_{3,2}, \dots, u_{n,2} \in \mathbb{R}^{n-2}$ such that

$$ Q_1^T u_{3,1} = \begin{pmatrix} * \\ u_{3,2} \end{pmatrix}, \quad Q_0^T u_{4,1} = \begin{pmatrix} * \\ u_{4,2} \end{pmatrix} \dots Q_0^T u_{n,1} = \begin{pmatrix} * \\ u_{n,2} \end{pmatrix}. $$

By lemma 11.1.2, we have $\text{Sp}(A_2) = \{\lambda_3, \dots, \lambda_n\}$ and for all $i \in \{3, \dots, n\}$, $u_{i,2} \in E_{\lambda_i}(A_2)$.

And so on, a sequence of orthogonal matrices $Q_0, Q_1, \dots, Q_{n-2}$ is constructed. We then set :

$$ Q := Q_0 \cdots Q_{n-2}. $$

We have :

$$ \begin{align*} T &:= Q^T A Q \\ &= Q_{n-1}^T \cdots Q_0^T A Q_0 \cdots Q_{n-1} \\ &= Q_{n-1}^T \cdots Q_1^T \begin{pmatrix} \lambda_1 & * \\ 0 & A_1 \end{pmatrix} Q_1 \cdots Q_{n-1} \\ &= Q_{n-1}^T \cdots Q_2^T \begin{pmatrix} \lambda_1 & * & * \\ 0 & \lambda_2 & * \\ 0 & 0 & A_2 \end{pmatrix} Q_2 \cdots Q_{n-1} \end{align*} $$
$$
\begin{align*}
&= Q_{n-2}^T \begin{pmatrix} \lambda_1 & & (*) \\ & \ddots & \\ (0) & & A_{n-2} \end{pmatrix} Q_{n-2} \\
&= \begin{pmatrix} \lambda_1 & (*) \\ & \ddots \\ (0) & & \lambda_n \end{pmatrix}
\end{align*}
$$

Thus we have $A = QTQ^T$ and :

$$
A^{-1} = QT^{-1}Q^{T}
$$

11.2 Example

Let's compute the inverse of :

$$
A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}
$$

using the Schur decomposition. We found in section 8 that the characteristic polynomial of A is given by :

$$
\chi_A(X) = -(X-1)^2 (X+3)
$$

and that :

$$
E_1(A) = \operatorname{Vect}\left(\begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}, \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix}\right) , \quad E_{-3}(A) = \operatorname{Vect}\left(\begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix}\right).
$$

Thus, $A$ is diagonalizable (hence trigonalisable) in $\mathbb{R}$. Let us set :

$$
u_{1,0} := \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix} , \quad u_{2,0} := \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} , \quad u_{3,0} := \begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix}.
$$

**First step.** We construct an orthonormal basis with the first vector being the normalized version of $u_{1,0}$ :

$$
\mathcal{B}_0 := \left( w_{1,0} := \frac{u_{1,0}}{\|u_{1,0}\|}, w_{2,0}, w_{3,0} \right) := \left( \frac{1}{\sqrt{5}} \begin{pmatrix} 2 \\ 1 \\ 0 \end{pmatrix}, \frac{1}{\sqrt{5}} \begin{pmatrix} -1 \\ 2 \\ 0 \end{pmatrix}, \begin{pmatrix} 0 \\ 0 \\ 1 \end{pmatrix} \right)
$$

Let us define :

$$
Q_0 := \frac{1}{\sqrt{5}} \begin{pmatrix} 2 & -1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & \sqrt{5} \end{pmatrix}
$$

We have :

$$
\begin{align*}
Q_0^T A Q_0 &= \frac{1}{\sqrt{5}} \begin{pmatrix} 2 & 1 & 0 \\ -1 & 2 & 0 \\ 0 & 0 & \sqrt{5} \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \frac{1}{\sqrt{5}} \begin{pmatrix} 2 & -1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & \sqrt{5} \end{pmatrix} \\
&= \frac{1}{5} \begin{pmatrix} 5 & -20 & 4\sqrt{5} \\ 0 & -10 & 3\sqrt{5} \\ 0 & 5\sqrt{5} & 0 \end{pmatrix} \\
&=: \begin{pmatrix} 1 & * \\ 0 & A_1 \end{pmatrix}
\end{align*}
$$
where :

$$A_1 := \frac{1}{5} \begin{pmatrix} -10 & 3\sqrt{5} \\ 5\sqrt{5} & 0 \end{pmatrix}.$$

**Second step.** We have :

$$Q_0^T u_{2,0} = \frac{1}{\sqrt{5}} \begin{pmatrix} 2 & 1 & 0 \\ -1 & 2 & 0 \\ 0 & 0 & \sqrt{5} \end{pmatrix} \begin{pmatrix} -1 \\ 0 \\ 1 \end{pmatrix} = \begin{pmatrix} -2/\sqrt{5} \\ 1/\sqrt{5} \\ 1 \end{pmatrix} = \begin{pmatrix} -2/\sqrt{5} \\ u_{2,1} \end{pmatrix}$$

$$Q_0^T u_{3,0} = \frac{1}{\sqrt{5}} \begin{pmatrix} 2 & 1 & 0 \\ -1 & 2 & 0 \\ 0 & 0 & \sqrt{5} \end{pmatrix} \begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix} = \begin{pmatrix} -4/\sqrt{5} \\ -3/\sqrt{5} \\ 1 \end{pmatrix} = \begin{pmatrix} -4/\sqrt{5} \\ u_{3,1} \end{pmatrix}$$

By lemma 11.1.2, we have that $Sp(A_1) = \{1, -3\}$ and

$$E_1(A_1) = \mathrm{Vect}(u_{2,1}) = \mathrm{Vect}\left(\frac{1}{\sqrt{5}}\right)$$

$$E_{-3}(A_1) = \mathrm{Vect}(u_{3,1}) = \mathrm{Vect}\left(\frac{-3}{\sqrt{5}}\right).$$

**Third step.** We construct an orthonormal basis with the first vector being the normalized version of $u_{2,1}$ :

$$ \begin{aligned} \mathcal{B}_1 &:= \left( w_{2,1} := \frac{u_{2,1}}{\|u_{2,1}\|}, w_{3,1} \right) \\ &= \left( \frac{1}{\sqrt{6/5}} \begin{pmatrix} 1/\sqrt{5} \\ 1 \end{pmatrix}, w_{3,0} \right) \\ &= \left( \begin{pmatrix} 1/\sqrt{6} \\ \sqrt{5/6} \end{pmatrix}, \begin{pmatrix} -\sqrt{5/6} \\ 1/\sqrt{6} \end{pmatrix} \right) \end{aligned} $$

Let us define :

$$Q_1 := \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/\sqrt{6} & -\sqrt{5/6} \\ 0 & \sqrt{5/6} & 1/\sqrt{6} \end{pmatrix}$$

Thus :

$$ \begin{aligned} T &:= Q_1^T \begin{pmatrix} 1 & * \\ 0 & A_1 \end{pmatrix} Q_1 \\ &= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/\sqrt{6} & \sqrt{5/6} \\ 0 & -\sqrt{5/6} & 1/\sqrt{6} \end{pmatrix} \frac{1}{5} \begin{pmatrix} 5 & -20 & 4\sqrt{5} \\ 0 & -10 & 3\sqrt{5} \\ 0 & 5\sqrt{5} & 0 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/\sqrt{6} & -\sqrt{5/6} \\ 0 & \sqrt{5/6} & 1/\sqrt{6} \end{pmatrix} \\ &= \frac{1}{5} \begin{pmatrix} 5 & -20 & 4\sqrt{5} \\ 0 & 5\sqrt{6}/2 & \frac{1}{2}\sqrt{6}\sqrt{5} \\ 0 & 5\sqrt{30}/2 & -5\sqrt{6}/2 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & \sqrt{6} & -\sqrt{6}/5 \\ 0 & \sqrt{6}/5 & \sqrt{6} \end{pmatrix} \\ &= \begin{pmatrix} 1 & 0 & 4\sqrt{6}/5 \\ 0 & 1 & -2/\sqrt{5} \\ 0 & 0 & -3 \end{pmatrix} \end{aligned} $$

and :

$$Q := Q_0 Q_1$$
$$ 
\begin{aligned}
&= \frac{1}{\sqrt{5}} \begin{pmatrix} 2 & -1 & 0 \\ 1 & 2 & 0 \\ 0 & 0 & \sqrt{5} \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/\sqrt{6} & -\sqrt{5/6} \\ 0 & \sqrt{5/6} & 1/\sqrt{6} \end{pmatrix} \\
&= \begin{pmatrix} 2/\sqrt{5} & -1/\sqrt{30} & 1/\sqrt{6} \\ 1/\sqrt{5} & 2/\sqrt{30} & -2/\sqrt{6} \\ 0 & \sqrt{5/6} & 1/\sqrt{6} \end{pmatrix}
\end{aligned}
$$

Since $T$ is upper triangular, its inverse is easily computed, we have by proposition A.6.2 :

$$ T^{-1} = \begin{pmatrix} 1 & 0 & 8/\sqrt{30} \\ 0 & 1 & -2/(3\sqrt{5}) \\ 0 & 0 & -1/3 \end{pmatrix}. $$

Since :

$$ 
\begin{aligned}
T &= Q_1^T \begin{pmatrix} 1 & \\ & A_1^* \end{pmatrix} Q_1 \\
&= Q_1^T Q_0^T A Q_0 Q_1 \\
&= Q^T A Q
\end{aligned}
$$

we have :

$$ 
\begin{aligned}
A^{-1} &= QT^{-1}Q^T \\
&= \begin{pmatrix} 2/\sqrt{5} & -1/\sqrt{30} & 1/\sqrt{6} \\ 1/\sqrt{5} & 2/\sqrt{30} & -2/\sqrt{6} \\ 0 & \sqrt{5/6} & 1/\sqrt{6} \end{pmatrix} \begin{pmatrix} 1 & 0 & 8/\sqrt{30} \\ 0 & 1 & -2/(3\sqrt{5}) \\ 0 & 0 & -1/3 \end{pmatrix} \begin{pmatrix} 2/\sqrt{5} & 1/\sqrt{5} & 0 \\ -1/\sqrt{30} & 2/\sqrt{30} & \sqrt{5/6} \\ 1/\sqrt{6} & -2/\sqrt{6} & 1/\sqrt{6} \end{pmatrix} \\
&= \begin{pmatrix} 2/\sqrt{5} & -1/\sqrt{30} & \sqrt{6}/2 \\ 1/\sqrt{5} & 2/\sqrt{30} & \sqrt{6}/3 \\ 0 & \sqrt{5/6} & -1/\sqrt{6} \end{pmatrix} \begin{pmatrix} 2/\sqrt{5} & 1/\sqrt{5} & 0 \\ -1/\sqrt{30} & 2/\sqrt{30} & \sqrt{5/6} \\ 1/\sqrt{6} & -2/\sqrt{6} & 1/\sqrt{6} \end{pmatrix} \\
&= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}
\end{aligned}
$$
# 12 Cholesky Decomposition

In section 10, we used the symmetric matrix $AA^T$, which allowed us to diagonalize with an orthogonal matrix.

There is information that we did not exploit : since the matrix $A$ is invertible, the matrix $AA^T$ is positive definite. Thanks to this new property, we obtain a new simple decomposition.

## 12.1 Introduction

Recall that a real symmetric matrix is diagonalizable and all its eigenvalues are real (see section 10). We begin with a definition.

**Definition 12.1.1**

Let $B \in \mathcal{M}_n(\mathbb{R})$ be a symmetric matrix. We say that $B$ is **positive definite** if all the eigenvalues of $B$ are strictly positive.

Thus, if $B$ is symmetric and positive definite, there exists a matrix $P \in O_n(\mathbb{R})$ such that :

$$P^T B P = \begin{pmatrix} \lambda_1 & & (0) \\ & \ddots & \\ (0) & & \lambda_n \end{pmatrix}$$

with $\lambda_1, \dots, \lambda_n > 0$ being the eigenvalues of $B$. Thus, $B$ is similar to a diagonal matrix with strictly positive elements on the diagonal (in particular, $B$ is invertible).

**Lemma 12.1.2**

(i) Let $A \in \text{GL}_n(\mathbb{R})$. Then the matrix $B := AA^T$ is a symmetric positive definite matrix.

(ii) Let $B \in \text{GL}_n(\mathbb{R})$ be a symmetric positive definite matrix. Then the principal minors of $B$ are all strictly positive.

**Proof.**

(i) We have :

$$\begin{aligned} B^T &= (AA^T)^T \\ &= (A^T)^T A^T \\ &= AA^T \\ &= B \end{aligned}$$

Let $\lambda$ be an eigenvalue of $B$ and $v$ an associated eigenvector. Let $u := A^T v$. Since $v$ is non-zero and $A^T$ is invertible, $u$ is non-zero i.e. $\|u\|^2 > 0$. Therefore :

$$\begin{aligned} \lambda \|v\|^2 &= \lambda v^T v \\ &= v^T B v \\ &= v^T A A^T v \\ &= (A^T v)^T A^T v \\ &= u^T u \end{aligned}$$
$$= \|u\|^2 > 0$$

i.e. we have $\lambda > 0$ because $\|v\|^2 > 0$ ($v$ is non-zero).

(ii) By proposition 10.1.3, there exists a matrix $P \in O_3(\mathbb{R})$ such that :

$$P^T B P = \begin{pmatrix} \lambda_1 & & (0) \\ & \lambda_2 & \\ (0) & & \lambda_3 \end{pmatrix}$$

with $\lambda_1, \lambda_2, \lambda_3 > 0$. We have :

$$B = P \begin{pmatrix} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \end{pmatrix} P^T \\ = \begin{pmatrix} p_{11} & p_{12} & p_{13} \\ p_{21} & p_{22} & p_{23} \\ p_{31} & p_{32} & p_{33} \end{pmatrix} \begin{pmatrix} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \end{pmatrix} \begin{pmatrix} p_{11} & p_{21} & p_{31} \\ p_{12} & p_{22} & p_{32} \\ p_{13} & p_{23} & p_{33} \end{pmatrix}$$

We have :

$$\begin{align*} \mu_1(B) &= b_{11} \\ &= \lambda_1 p_{11}^2 + \lambda_2 p_{12}^2 + \lambda_3 p_{13}^2 > 0 \end{align*} \\ \mu_3(B) &= \det(B) \\ &= \det \begin{pmatrix} \lambda_1 & 0 & 0 \\ 0 & \lambda_2 & 0 \\ 0 & 0 & \lambda_3 \end{pmatrix} P^T \\ &= \det(P)^2 \lambda_1 \lambda_2 \lambda_3 > 0 \end{align*}$$

Since $P$ is invertible, the last two rows are not collinear i.e. one of the following three determinants is non-zero :

$$\Delta_{33}(P), \Delta_{32}(P), \Delta_{31}(P).$$

Thus :

$$\begin{align*} \mu_2(B) &= \begin{vmatrix} b_{11} & b_{12} \\ b_{12} & b_{22} \end{vmatrix} \\ &= b_{11}b_{22} - b_{12}^2 \\ &= (\lambda_1 p_{11}^2 + \lambda_2 p_{12}^2 + \lambda_3 p_{13}^2)(\lambda_1 p_{21}^2 + \lambda_2 p_{22}^2 + \lambda_3 p_{23}^2) - (\lambda_1 p_{11} p_{21} + \lambda_2 p_{12} p_{22} + \lambda_3 p_{13} p_{23})^2 \\ &= \lambda_1 \lambda_2 (p_{11}^2 p_{22}^2 + p_{12}^2 p_{21}^2 - 2p_{11}p_{12}p_{21}p_{22}) + \lambda_1 \lambda_3 (p_{11}^2 p_{23}^2 + p_{13}^2 p_{21}^2 - 2p_{11}p_{13}p_{21}p_{23}) \\ &\quad + \lambda_2 \lambda_3 (p_{12}^2 p_{23}^2 + p_{13}^2 p_{22}^2 - 2p_{12}p_{13}p_{22}p_{23}) \\ &= \lambda_1 \lambda_2 (p_{11}p_{22} - p_{12}p_{21})^2 + \lambda_1 \lambda_3 (p_{11}p_{23} - p_{13}p_{21})^2 + \lambda_2 \lambda_3 (p_{12}p_{23} - p_{13}p_{22})^2 \\ &= \lambda_1 \lambda_2 \Delta_{33}(P)^2 + \lambda_1 \lambda_3 \Delta_{32}(P)^2 + \lambda_2 \lambda_3 \Delta_{31}(P)^2 > 0 \end{align*}$$

We then deduce the Cholesky decomposition.

<figure><img src="image_9.png" /></figure>
### Proposition 12.1.3: Cholesky Decomposition

Let $B \in GL_n(\mathbb{R})$ be a symmetric positive definite matrix. Then there exists a unique real lower triangular matrix $L$ with strictly positive diagonal elements such that :

$$B = LL^T.$$

**Proof.** Let us prove the result in the case $n := 3$. Let :

$$B := \begin{pmatrix} b_{11} & b_{12} & b_{13} \\ b_{12} & b_{22} & b_{23} \\ b_{13} & b_{23} & b_{33} \end{pmatrix}.$$

Let the matrix be :

$$L := \begin{pmatrix} a & 0 & 0 \\ b & c & 0 \\ d & e & f \end{pmatrix}$$

with $a, c, f > 0$. Since :

$$LL^T = \begin{pmatrix} a & 0 & 0 \\ b & c & 0 \\ d & e & f \end{pmatrix} \begin{pmatrix} a & b & d \\ 0 & c & e \\ 0 & 0 & f \end{pmatrix} = \begin{pmatrix} a^2 & ab & ad \\ ab & b^2 + c^2 & bd + ce \\ ad & bd + ce & d^2 + e^2 + f^2 \end{pmatrix}$$

we have the equivalences :

$$B = LL^T \iff \begin{pmatrix} b_{11} & b_{12} & b_{13} \\ b_{12} & b_{22} & b_{23} \\ b_{13} & b_{23} & b_{33} \end{pmatrix} = \begin{pmatrix} a^2 & ab & ad \\ ab & b^2 + c^2 & bd + ce \\ ad & bd + ce & d^2 + e^2 + f^2 \end{pmatrix}$$

$$\iff \begin{cases} a^2 = b_{11} \\ ab = b_{12} \\ ad = b_{13} \\ b^2 + c^2 = b_{22} \\ bd + ce = b_{23} \\ d^2 + e^2 + f^2 = b_{33} \end{cases}$$

$$\iff \begin{cases} a = \sqrt{b_{11}} = \sqrt{\mu_1} & \text{because } a, c, f > 0 \\ b = \frac{b_{12}}{a} = \frac{b_{12}}{\sqrt{\mu_1}} \\ d = \frac{b_{13}}{a} = \frac{b_{13}}{\sqrt{\mu_1}} \end{cases}$$

$$\iff \begin{cases} c = \sqrt{b_{22} - b^2} = \sqrt{\frac{b_{22}b_{11} - b_{12}^2}{\mu_1}} = \sqrt{\frac{\mu_2}{\mu_1}} \\ e = \frac{b_{23} - bd}{c} = \frac{b_{11}b_{23} - b_{12}b_{13}}{\sqrt{\mu_1\mu_2}} \\ f = \sqrt{b_{33} - d^2 - e^2} = \frac{\sqrt{b_{33}\mu_2 - \frac{\mu_2b_{12} - (b_{11}b_{23} - b_{12}b_{13})^2}{\mu_1}}}{\sqrt{\mu_2}} = \frac{\sqrt{\det(B)}}{\sqrt{\mu_2}} \end{cases}$$
$$ \iff L = \frac{1}{\sqrt{\mu_1 \mu_2}} \begin{pmatrix} \mu_2 & 0 & 0 \\ \mu_2 b_{12} & \mu_2 b_{13} & 0 \\ \mu_2 & b_{11} b_{23} - b_{12} b_{13} & \sqrt{\mu_1 \det(B)} \end{pmatrix} $$

Thus, the existence and uniqueness follow from the equivalence.

<figure><img src="image_10.png" /></figure>

We then obtain a simple technique for calculating the inverse of a matrix A. Let :

$$ B := AA^T $$

Then B is symmetric positive definite. So there exists a unique real lower triangular matrix L with strictly positive diagonal elements such that :

$$ B = LL^T. $$

Since :

$$ A^{-1} = A^T B^{-1} $$

we have :

$$ A^{-1} = A^T (L^{-1})^T L^{-1} \qquad (12.1.1) $$

## 12.2 Example

Let's calculate the inverse of :

$$ A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} $$

using the Cholesky decomposition and the formula (12.1.1). We have :

$$ \begin{aligned} B &:= AA^T \\ &= \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 2 & 2 & -1 \\ -2 & -3 & 2 \\ 1 & 2 & 0 \end{pmatrix} \\ &= \begin{pmatrix} 9 & 12 & -6 \\ 12 & 17 & -8 \\ -6 & -8 & 5 \end{pmatrix} \end{aligned} $$

Let the matrix :

$$ L := \begin{pmatrix} a & 0 & 0 \\ b & c & 0 \\ d & e & f \end{pmatrix} $$

with a, c, f > 0. We have :

$$ \begin{align*} B = LL^T &\iff \begin{pmatrix} 9 & 12 & -6 \\ 12 & 17 & -8 \\ -6 & -8 & 5 \end{pmatrix} = \begin{pmatrix} a^2 & ab & ad \\ ab & b^2 + c^2 & bd + ce \\ ad & bd + ce & d^2 + e^2 + f^2 \end{pmatrix} \\ &\implies \begin{cases} a = 3, b = 4, d = -2 \\ c^2 + b^2 = 17, bd + ce = -8 \\ 4 + e^2 + f^2 = 5 \end{cases} \\ &\iff \begin{cases} a = 3, b = 4, d = -2 \\ c = 1, e = 0 \\ f = 1 \end{cases} \end{align*} $$
$$ \iff L = \begin{pmatrix} 3 & 0 & 0 \\ 4 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix} $$

The inverse of $L$ can be easily calculated using one of the methods previously described since $L$ is a lower triangular matrix. Using the formula from proposition A.6.1 for the inverse of a lower triangular matrix, we have :

$$ L^{-1} = \frac{1}{3} \begin{pmatrix} 1 & 0 & 0 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} $$

Thus, we have :

$$ \begin{aligned} B^{-1} &= (L^{-1})^T L^{-1} \\ &= \frac{1}{9} \begin{pmatrix} 1 & -4 & 2 \\ 0 & 3 & 0 \\ 0 & 0 & 3 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 7 & -4 & 2 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} \end{aligned} $$

And so we have :

$$ \begin{aligned} A^{-1} &= A^T B^{-1} \\ &= \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ -2 & -3 & 2 \\ 1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 7 & -4 & 2 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{aligned} $$
# 13 QR Decomposition and Gram-Schmidt Orthonormalization

In this section, we present the QR decomposition of an invertible matrix using the Gram-Schmidt orthonormalization process to find the orthogonal matrix $Q$. In the following section 14 and in the appendix D, we will see two other methods for calculating a matrix $Q$.

## 13.1 Introduction

### Definition 13.1.1

Let $(u_1, \dots, u_k)$ be a linearly independent family in $\mathbb{R}^n$ with $k \in \{1, \dots, n\}$ (i.e., no zero vectors).

(i) We say that $(u_1, \dots, u_k)$ is an **orthogonal family in $\mathbb{R}^n$ if $^a$ :

$$ \forall i, j \in \{1, \dots, n\}, \quad i \neq j \implies \langle u_i, u_j \rangle = 0. $$

We further say that the family is an **orthogonal basis** if it is also a basis of $\mathbb{R}^n$ (i.e. $k = n$).

(ii) We say that $(u_1, \dots, u_k)$ is an **orthonormal family in $\mathbb{R}^n$ if :

$$ \forall i, j \in \{1, \dots, n\}, \quad \langle u_i, u_j \rangle = \delta_{i,j}. $$

We further say that the family is an **orthonormal basis** if it is also a basis of $\mathbb{R}^n$ (i.e. $k = n$).

We then have the Gram-Schmidt orthonormalization process which constructs an orthonormal basis (for the usual inner product) from a given basis.

### Lemma 13.1.2: Gram-Schmidt Orthonormalization

Let $(u_1, \dots, u_n)$ be a basis of $\mathbb{R}^n$. Define the two families $(v_1, \dots, v_n)$ and $(w_1, \dots, w_n)$ by the following recurrence :

$$ v_1 := u_1 $$

$$ w_1 := \frac{v_1}{\|v_1\|} $$

$$ v_k := u_k - \sum_{i=1}^{k-1} \langle u_k, w_i \rangle w_i \quad (k \in \{2, \dots, n\}) $$

$$ w_k := \frac{v_k}{\|v_k\|} \quad (k \in \{2, \dots, n\}) $$

Then we have the following results :

(i) The family $(v_1, \dots, v_n)$ is an orthogonal basis of $\mathbb{R}^n$;

(ii) The family $(w_1, \dots, w_n)$ is an orthonormal basis of $\mathbb{R}^n$;

(iii) For all $k \in \{1, \dots, n\}$ :

$$ \mathrm{Vect}_{\mathbb{R}}(u_1, \dots, u_k) = \mathrm{Vect}_{\mathbb{R}}(v_1, \dots, v_k) = \mathrm{Vect}_{\mathbb{R}}(w_1, \dots, w_k). $$

We then set :

$$ (w_1, \dots, w_n) := \mathrm{GS}(u_1, \dots, u_n). $$

<footnote>a. One could more generally define an orthogonal family with zero vectors.</footnote>
**Proof.** Let us prove the lemma for $n := 3$. We have :

$$v_1 := u_1$$

$$w_1 := \frac{u_1}{\|u_1\|}$$

$$v_2 := u_2 - \langle u_2, w_1 \rangle w_1$$

$$w_2 := \frac{v_2}{\|v_2\|}$$

$$v_3 := u_3 - \langle u_3, w_2 \rangle w_2 - \langle u_3, w_1 \rangle w_1$$

$$w_3 := \frac{v_3}{\|v_3\|}$$

Let us start with a remark :

(*) For any non-zero vector $x \in \mathbb{R}^n$, $x/\|x\|$ is **unitary** (i.e. of norm 1) because :

$$\left\| \frac{x}{\|x\|} \right\| = \frac{\|x\|}{\|x\|} = 1.$$

Let us show that the family $(v_1, v_2, v_3)$ is linearly independent (i.e., no zero vector).

(1) By definition, we have :

$$\mathrm{Vect}_{\mathbb{R}}(u_1) = \mathrm{Vect}_{\mathbb{R}}(v_1) = \mathrm{Vect}_{\mathbb{R}}(w_1).$$

(2) By linearity of the inner product :

$$\begin{align*}
\langle v_1, v_2 \rangle &= \langle v_1, u_2 - \langle u_2, w_1 \rangle w_1 \rangle \\
&= \langle v_1, u_2 \rangle - \langle u_2, w_1 \rangle \langle v_1, w_1 \rangle \\
&= \langle v_1, u_2 \rangle - \langle u_2, \frac{v_1}{\|v_1\|} \rangle \langle v_1, \frac{v_1}{\|v_1\|} \rangle \\
&= \langle v_1, u_2 \rangle - \langle u_2, v_1 \rangle \\
&= 0
\end{align*}$$

And by definition :

$$v_1, v_2 \in \mathrm{Vect}_{\mathbb{R}}(u_1, u_2, w_1) = \mathrm{Vect}_{\mathbb{R}}(u_1, u_2)$$

i.e., we have :

$$\mathrm{Vect}_{\mathbb{R}}(v_1, v_2) \subset \mathrm{Vect}_{\mathbb{R}}(u_1, u_2).$$

Let us show that $(v_1, v_2)$ is a linearly independent family. Suppose by contradiction that $v_2$ is collinear to $v_1$ i.e., $v_2 = \alpha v_1$ ($\alpha \in \mathbb{R}$). Then we have :

$$\begin{align*}
u_2 &= v_2 + \langle u_2, w_1 \rangle w_1 \\
&= \left( \alpha + \frac{\langle u_2, w_1 \rangle}{\|u_1\|} \right) v_1
\end{align*}$$

i.e., the family $(u_1, u_2)$ is linearly dependent (which is a contradiction). Therefore, the family $(v_1, v_2)$ is linearly independent. Thus, the family $(v_1, v_2)$ is orthogonal and :

$$\mathrm{Vect}_{\mathbb{R}}(v_1, v_2) = \mathrm{Vect}_{\mathbb{R}}(u_1, u_2).$$

Since the family $(v_1, v_2)$ is linearly independent, it does not contain any zero vectors. Therefore, the family $(w_1, w_2)$ is **normal** (i.e. contains only unit vectors) by the remark (*) and linearly independent (because if the family $(w_1, w_2)$ were dependent i.e., $w_1$ and $w_2$ were collinear, then the family $(v_1, v_2)$ would be dependent, which is a contradiction). Therefore, the family $(w_1, w_2)$ is orthonormal. Moreover, we have :

$$w_1, w_2 \in \mathrm{Vect}_{\mathbb{R}}(u_1, v_2) = \mathrm{Vect}_{\mathbb{R}}(v_1, v_2)$$

i.e., since the family $(w_1, w_2)$ is linearly independent, we have :

$$\mathrm{Vect}_{\mathbb{R}}(w_1, w_2) = \mathrm{Vect}_{\mathbb{R}}(v_1, v_2).$$
(3) For $i \in \{1, 2\}$, by linearity of the inner product :

$$
\begin{align*}
\langle v_i, v_3 \rangle &= \langle v_i, u_3 - \langle u_3, w_2 \rangle w_2 - \langle u_3, w_1 \rangle w_1 \rangle \\
&= \langle v_i, u_3 \rangle - \langle u_3, w_2 \rangle \langle v_i, w_2 \rangle - \langle u_3, w_1 \rangle \langle v_i, w_1 \rangle \\
&= \langle \|v_i\| w_i, u_3 \rangle - \langle u_3, w_2 \rangle \langle \|v_i\| w_i, w_2 \rangle - \langle u_3, w_1 \rangle \langle \|v_i\| w_i, w_1 \rangle \\
&= \|v_i\| \langle w_i, u_3 \rangle - \langle u_3, w_2 \rangle \|v_i\| \delta_{i2} - \langle u_3, w_1 \rangle \|v_i\| \delta_{i1} \\
&= 0
\end{align*}
$$

And by definition :

$$
v_1, v_2, v_3 \in \mathrm{Vect}_{\mathbb{R}}(u_1, u_2, w_1, u_3, w_2, w_1) = \mathrm{Vect}_{\mathbb{R}}(u_1, u_2, u_3)
$$

i.e., we have :

$$
\mathrm{Vect}_{\mathbb{R}}(v_1, v_2, v_3) \subset \mathrm{Vect}_{\mathbb{R}}(u_1, u_2, u_3).
$$

Let us show that $(v_1, v_2, v_3)$ is a linearly independent family. Suppose by contradiction that $v_3$ is coplanar with $v_1$ and $v_2$ i.e., $v_3 = \alpha v_1 + \beta v_2$ $(\alpha, \beta \in \mathbb{R})$. Then we have :

$$
\begin{align*}
u_3 &= v_3 + \langle u_3, w_2 \rangle w_2 + \langle u_3, w_1 \rangle w_1 \\
&= \left( \alpha + \frac{\langle u_2, w_1 \rangle}{\|u_1\|} \right) u_1
\end{align*}
$$

i.e., the family $(u_1, u_2, u_3)$ is linearly dependent (which is a contradiction). Therefore, the family $(v_1, v_2, v_3)$ is linearly independent. Thus, the family $(v_1, v_2, v_3)$ is orthogonal and :

$$
\mathrm{Vect}_{\mathbb{R}}(v_1, v_2, v_3) = \mathrm{Vect}_{\mathbb{R}}(u_1, u_2, v_3).
$$

Since the family $(v_1, v_2, v_3)$ is linearly independent, it does not contain any zero vectors. Therefore, the family $(w_1, w_2, w_3)$ is normal by the remark (*) and linearly independent (because if the family $(w_1, w_2, w_3)$ were dependent, then the family $(v_1, v_2, v_3)$ would be dependent, which is a contradiction). Therefore, the family $(w_1, w_2, w_3)$ is orthonormal. Moreover, we have :

$$
w_1, w_2, w_3 \in \mathrm{Vect}_{\mathbb{R}}(u_1, v_2, v_3) = \mathrm{Vect}_{\mathbb{R}}(v_1, v_2, v_3)
$$

i.e., since the family $(w_1, w_2, w_3)$ is linearly independent, we have :

$$
\mathrm{Vect}_{\mathbb{R}}(w_1, w_2, w_3) = \mathrm{Vect}_{\mathbb{R}}(v_1, v_2, v_3).
$$

The orthonormalization process constructs two bases of vectors : the basis $(v_1, \dots, v_n)$ which is orthogonal, and the basis $(w_1, \dots, w_n)$ which is orthonormal. An element from each basis is constructed at each step. One could have separated the constructions and first built the basis $(v_1, \dots, v_n)$ (the **orthogonalization**) and then the basis $(w_1, \dots, w_n)$ (the **normalization**).

**Proposition 13.1.3**

Let :

$$
A := (u_1 | \cdots | u_n).
$$

Let us set :

$$
(w_1, \dots, w_n) := \mathrm{GS}(u_1, \dots, u_n) , \quad Q := (w_1 | \cdots | w_n) , \quad R := Q^T A.
$$

Then the matrix $Q$ is orthonormal and the matrix $R$ is an invertible upper triangular matrix.
Thus we have :

$$A^{-1} = R^{-1}Q^{-1} = R^{-1}Q^{T}.$$

**Proof.** We prove the result in the case $n := 3$. Let :

$$A := (u_1 \mid u_2 \mid u_3) \in \mathrm{GL}_3(\mathbb{R}).$$

Construct the orthonormal basis $(w_1, w_2, w_3)$ associated with $(u, v, w)$ using the Gram-Schmidt process. We have :

$$v_1 := u_1$$

$$w_1 := \frac{u_1}{\|u_1\|}$$

$$v_2 := u_2 - \langle u_2, w_1 \rangle w_1$$

$$w_2 := \frac{v_2}{\|v_2\|}$$

$$v_3 := u_3 - \langle u_3, w_2 \rangle w_2 - \langle u_3, w_1 \rangle w_1$$

$$w_3 := \frac{v_3}{\|v_3\|}$$

Let :

$$Q := (w_1 \mid w_2 \mid w_3).$$

Since the basis $(w_1, w_2, w_3)$ is orthonormal, we have :

$$
\begin{align*}
Q^T Q &= \begin{pmatrix} w_1^T \\ w_2^T \\ w_3^T \end{pmatrix} (w_1 \mid w_2 \mid w_3) \\
&= \begin{pmatrix} w_1^T w_1 & w_1^T w_2 & w_1^T w_3 \\ w_2^T w_1 & w_2^T w_2 & w_2^T w_3 \\ w_3^T w_1 & w_3^T w_2 & w_3^T w_3 \end{pmatrix} \\
&= \begin{pmatrix} \langle w_1, w_1 \rangle & \langle w_1, w_2 \rangle & \langle w_1, w_3 \rangle \\ \langle w_2, w_1 \rangle & \langle w_2, w_2 \rangle & \langle w_2, w_3 \rangle \\ \langle w_3, w_1 \rangle & \langle w_3, w_2 \rangle & \langle w_3, w_3 \rangle \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \\
&= I_3
\end{align*}
$$

And we have :

$$
\begin{align*}
R &= Q^T A \\
&= \begin{pmatrix} w_1^T \\ w_2^T \\ w_3^T \end{pmatrix} (u_1 \mid u_2 \mid u_3) \\
&= \begin{pmatrix} w_1^T u_1 & w_1^T u_2 & w_1^T u_3 \\ w_2^T u_1 & w_2^T u_2 & w_2^T u_3 \\ w_3^T u_1 & w_3^T u_2 & w_3^T u_3 \end{pmatrix} \\
&= \begin{pmatrix} \langle w_1, u_1 \rangle & \langle w_1, u_2 \rangle & \langle w_1, u_3 \rangle \\ \langle w_2, u_1 \rangle & \langle w_2, u_2 \rangle & \langle w_2, u_3 \rangle \\ \langle w_3, u_1 \rangle & \langle w_3, u_2 \rangle & \langle w_3, u_3 \rangle \end{pmatrix} \\
&= \begin{pmatrix} \langle w_1, u_1 \rangle & \langle w_1, u_2 \rangle & \langle w_1, u_3 \rangle \\ 0 & \langle w_2, u_2 \rangle & \langle w_2, u_3 \rangle \\ 0 & 0 & \langle w_3, u_3 \rangle \end{pmatrix}
\end{align*}
$$
Hence the results.

We can express the family $(u_1, \dots, u_n)$ in terms of the orthonormal family $(w_1, \dots, w_n)$. For $n := 3$, we have :

$$
\begin{align*}
u_1 &= \|u_1\| w_1 = \langle u_1, w_1 \rangle w_1 \\
u_2 &= v_2 + \langle u_2, w_1 \rangle w_1 = \|v_2\| w_2 + \langle u_2, w_1 \rangle w_1 \\
&= \langle v_2, w_2 \rangle w_2 + \langle u_2, w_1 \rangle w_1 \\
&= \langle u_2 - \langle u_2, w_1 \rangle w_1, w_2 \rangle w_2 + \langle u_2, w_1 \rangle w_1 \\
&= \langle u_2, w_2 \rangle w_2 + \langle u_2, w_1 \rangle w_1 \\
u_3 &= v_3 + \langle u_3, w_2 \rangle w_2 + \langle u_3, w_1 \rangle w_1 \\
&= \|v_3\| w_3 + \langle u_3, w_2 \rangle w_2 + \langle u_3, w_1 \rangle w_1 \\
&= \langle v_3, w_3 \rangle w_3 + \langle u_3, w_2 \rangle w_2 + \langle u_3, w_1 \rangle w_1 \\
&= \langle u_3 - \langle u_3, w_2 \rangle w_2 - \langle u_3, w_1 \rangle w_1, w_3 \rangle w_3 + \langle u_3, w_2 \rangle w_2 + \langle u_3, w_1 \rangle w_1 \\
&= \langle u_3, w_3 \rangle w_3 + \langle u_3, w_2 \rangle w_2 + \langle u_3, w_1 \rangle w_1
\end{align*}
$$

13.2 Example

Calculate the inverse of :

$$
A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}
$$

using the *QR* decomposition with the matrix *Q* computed by the Gram-Schmidt orthonormalization process on
the columns of *A*. We denote the columns of *A* by :

$$
(u_1 | u_2 | u_3) := A = \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}
$$

Construct the orthonormal basis $(w_1, w_2, w_3)$ associated with $(u_1, u_2, u_3)$. We have :

$$
\begin{align*}
w_1 &:= \frac{u_1}{\|u_1\|} = \frac{1}{3} \begin{pmatrix} 2 \\ 2 \\ -1 \end{pmatrix} \\
v_2 : u_2 - \langle u_2, w_1 \rangle w_1 &= \begin{pmatrix} -2 \\ -3 \\ 2 \end{pmatrix} - \frac{-12}{9} \begin{pmatrix} 2 \\ 2 \\ -1 \end{pmatrix} = \frac{1}{3} \begin{pmatrix} 2 \\ -1 \\ 2 \end{pmatrix} \\
w_2 &:= \frac{v_2}{\|v_2\|} = v_2 = \frac{1}{3} \begin{pmatrix} 2 \\ -1 \\ 2 \end{pmatrix} \\
v_3 &:= u_3 - \langle u_3, w_1 \rangle w_1 - \langle u_3, w_2 \rangle w_2 = \begin{pmatrix} 1 \\ 2 \\ 0 \end{pmatrix} - \frac{6}{9} \begin{pmatrix} 2 \\ 2 \\ -1 \end{pmatrix} = \frac{1}{3} \begin{pmatrix} -1 \\ 2 \\ 2 \end{pmatrix} \\
w_3 &:= \frac{v_3}{\|v_3\|} = w' = \frac{1}{3} \begin{pmatrix} -1 \\ 2 \\ 2 \end{pmatrix}
\end{align*}
$$

Let us set :

$$
Q := (w_1 | w_2 | w_3) = \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}
$$
We have (it's unnecessary to calculate) :

$$
\begin{align*}
QQ^T &= \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \\
&= I_3
\end{align*}
$$

i.e., $Q$ is orthogonal (and even symmetric here so $Q^{-1} = Q^T = Q$ i.e., $Q$ is **involutory**). We have :

$$
\begin{align*}
R &= Q^T A \\
&= QA \\
&= \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \\
&= \begin{pmatrix} 3 & -4 & 2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}.
\end{align*}
$$

The inverse of $R$ can be easily computed using one of the methods described earlier as $R$ is an upper triangular matrix. For example, using the method of the cofactor matrix. Since :

$$
\mathrm{Com}(R) = \begin{pmatrix} 1 & 0 & 0 \\ 4 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix}
$$

we have :

$$
R^{-1} = \frac{1}{\det(R)} \mathrm{Com}(R)^T = \frac{1}{3} \begin{pmatrix} 1 & 4 & -2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}
$$

Thus, since $A = QR$, we have :

$$
\begin{align*}
A^{-1} &= R^{-1}Q^{-1} \\
&= R^{-1}Q^T \\
&= \frac{1}{3} \begin{pmatrix} 1 & 4 & -2 \\ 0 & 3 & 0 \\ 0 & 0 & 3 \end{pmatrix} \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \\
&= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}
\end{align*}
$$
# 14 Decomposition RQ and Householder Matrices

This section serves more as an opportunity to explore a new linear algebra technique. The QR decomposition uses the columns of the matrix, while the RQ decomposition uses the rows of the matrix. In this section, we will use Householder matrices to find the matrix Q.<fnref n="7" />

In the previous section 13 and in the appendix D, you will find two other methods for computing a matrix Q.

## 14.1 Introduction

### Definition 14.1.1

Let $v \in \mathbb{R}^n$ be non-zero. The Householder matrix associated with $v$ is:

$$H_v := I_n - 2 \frac{vv^T}{\|v\|^2}.$$

We have:

$$H_v = I_n - 2 \frac{vv^T}{v^T v}.$$

Householder matrices satisfy simple properties that allow for a QR decomposition.

### Lemma 14.1.2

(i) Let $v \in \mathbb{R}^n$ be non-zero. Then $H_v$ is symmetric and involutory (thus orthogonal).

(ii) Let $x, y \in \mathbb{R}^n$ such that $x \neq y$ and $\|x\| = \|y\|$. Then we have:

$$H_{x-y}(x) = y.$$

### Proof.

(i) We have:

$$ \begin{aligned} (H_v)^T &= \left( I_n - 2 \frac{vv^T}{\|v\|^2} \right)^T \\ &= I_n - 2 \frac{vv^T}{\|v\|^2} \\ &= H_v \end{aligned} $$

$$ \begin{aligned} H_v^2 &= \left( I_n - 2 \frac{vv^T}{\|v\|^2} \right)^2 \\ &= I_n - 4 \frac{vv^T}{\|v\|^2} + 4 \frac{v(v^T v)v^T}{\|v\|^4} \\ &= I_n - 4 \frac{vv^T}{\|v\|^2} + 4 \frac{vv^T}{\|v\|^2} \\ &= I_n \end{aligned} $$

(ii) Since $x^T x = y^T y$, we have:

$$\|x - y\|^2 = (x - y)^T (x - y)$$

<footnote n="7">7. The RQ decomposition is simply a QR decomposition applied to the transpose. The idea of this text is to practice linear algebra techniques; you can view this section as an additional trick.</footnote>
$$
\begin{align*}
&= x^T x - 2x^T y + y^T y \\
&= x^T x - 2x^T y + x^T x \\
&= 2x^T (x - y)
\end{align*}
$$

Thus we have :

$$
\begin{align*}
H_{x-y}(x) &= x - 2 \frac{(x-y)(x-y)^T x}{\|x-y\|^2} \\
&= x - \frac{2x^T(x-y)}{2x^T(x-y)}(x-y) \\
&= x - (x-y) \\
&= y
\end{align*}
$$

Thus, $H_v$ is the matrix of the orthogonal reflection with respect to $v^\perp$ (the hyperplane orthogonal to $v$). We then deduce a method for computing $Q$ in the $QR$ decomposition using Householder matrices.

Proposition 14.1.3

Let :

$$
A := (u_1 | * | \cdots | *) \in \mathrm{GL}_n(\mathbb{R})
$$

Then the matrix $H_{u_1-\|u_1\|e_1} A$ is of the form :

$$
\left(
\begin{array}{cc}
\|u_1\| & * \\
0 & A_1
\end{array}
\right)
$$

with $A_1 \in \mathrm{GL}_{n-1}(\mathbb{R})$.

**Proof.** Let $A$ be the matrix defined by columns as :

$$
A := (u_1 | * | \cdots | *) \in \mathrm{GL}_n(\mathbb{R}).
$$

Then by point (ii) of the previous lemma, we have :

$$
\begin{align*}
H_{u_1 - \|u_1\| e_1} A &= (H_{u_1 - \|u_1\| e_1} u \mid * \mid \cdots \mid *) \\
&= (\|u_1\| e_1 \mid * \mid \cdots \mid *) \\
&:= \begin{pmatrix} \|u_1\| & * \\ 0 & A_1 \end{pmatrix}
\end{align*}
$$

with $A_1 \in \mathrm{GL}_{n-1}(\mathbb{R})$.

We then deduce a method for computing the matrix $Q$. We reapply the proposition to the matrix :

$$
A_1 := (u_2 | * | \cdots | *) \in \mathrm{GL}_{n-1}(\mathbb{R})
$$

Then the matrix $H_{u_2-\|u_2\|e_1^{(n-1)}} A_1$ (where $e_1^{(n-1)}$ is the first vector of the canonical basis of $\mathbb{R}^{n-1}$, with the exponent "(n - 1)" indicating the dimension of the vector) is of the form :

$$
\left(
\begin{array}{cc}
\|u_2\| & * \\
0 & A_2
\end{array}
\right) := H_{u_2 - \|u_2\| e_1^{(n-1)}} A_1
$$
with $A_2 \in \text{GL}_{n-2}(\mathbb{R})$. We construct successive orthogonal matrices :

$$Q_1 := H_{u_1-\|u_1\|e_1^{(n)}} , \quad Q_2 := \begin{pmatrix} 1 & 0 \\ 0 & H_{u_2-\|u_2\|e_1^{(n-1)}} \end{pmatrix} , \dots , \quad Q_k := \begin{pmatrix} I_k & 0 \\ 0 & H_{u_k-\|u_k\|e_1^{(n-k+1)}} \end{pmatrix} \quad (k \in \{1, \dots, n\}).$$

Let

$$Q := Q_n \cdots Q_1$$

We then have successively :

$$ \begin{align*} QA &= Q_n \cdots Q_1 A \\ &= Q_n \cdots Q_2 \begin{pmatrix} \|u_1\| & * \\ 0 & A_1 \end{pmatrix} \\ &= Q_n \cdots Q_3 \begin{pmatrix} \|u_1\| & * & * \\ 0 & \|u_2\| & * \\ 0 & 0 & A_2 \end{pmatrix} \\ &= \dots \\ &= \begin{pmatrix} \|u_1\| & * & * \\ 0 & \ddots & * \\ 0 & 0 & \|u_n\| \end{pmatrix} \end{align*} $$

By taking the transpose, we obtain a similar result but by decomposing $A$ according to its rows.

### Corollary 14.1.4

Let :

$$A := \begin{pmatrix} u_1 \\ * \\ \vdots \\ * \end{pmatrix} \in \text{GL}_n(\mathbb{R})$$

Then the matrix $AH_{u_1-\|u_1\|e_1}$ is of the type :

$$\begin{pmatrix} \|u_1\| & 0 \\ * & A_1 \end{pmatrix}$$

with $A_1 \in \text{GL}_{n-1}(\mathbb{R})$.

We can deduce once again a method to calculate the matrix $Q$. Reapply the proposition to the matrix :

$$A_1 := \begin{pmatrix} u_2 \\ * \\ \vdots \\ * \end{pmatrix} \in \text{GL}_{n-1}(\mathbb{R})$$

Then the matrix $A_1 H_{u_2-\|u_2\|e_1^{(n-1)}}$ is of the type :

$$\begin{pmatrix} \|u_2\| & 0 \\ * & A_2 \end{pmatrix} := A_1 H_{u_2-\|u_2\|e_1^{(n-1)}}$$

with $A_2 \in \text{GL}_{n-2}(\mathbb{R})$. We construct successive orthogonal matrices :

$$Q_1 := H_{u_1-\|u_1\|e_1^{(n)}} , \quad Q_2 := \begin{pmatrix} 1 & 0 \\ 0 & H_{u_2-\|u_2\|e_1^{(n-1)}} \end{pmatrix} , \dots , \quad Q_k := \begin{pmatrix} I_k & 0 \\ 0 & H_{u_k-\|u_k\|e_1^{(n-k+1)}} \end{pmatrix} \quad (k \in \{1, \dots, n\}).$$
Let

$$Q := Q_1 \cdots Q_n$$

We have successively :

$$
\begin{align*}
AQ &= AQ_1 \cdots Q_n \\
&= \begin{pmatrix} \|u_1\| & 0 \\ A_1 & \end{pmatrix} Q_2 \cdots Q_n \\
&= \begin{pmatrix} \|u_1\| & 0 & 0 \\ * & \|u_2\| & 0 \\ * & * & A_2 \end{pmatrix} Q_3 \cdots Q_n \\
&= \dots \\
&= \begin{pmatrix} \|u_1\| & 0 & 0 \\ * & \ddots & 0 \\ * & * & \|u_n\| \end{pmatrix}
\end{align*}
$$

## 14.2 Example

Calculate the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

using the QR decomposition (here we will do it in the opposite direction, which we can call RQ decomposition) with the matrix Q calculated using Householder matrices. As indicated at the end of the previous subsection, we apply the results to the rows of A (thus using the previous corollary).

Let :

$$A := \begin{pmatrix} u \\ v \\ w \end{pmatrix} := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}.$$

Let :

$$u' := u - \|u\| e_1 = \begin{pmatrix} 2 \\ -2 \\ 1 \end{pmatrix} - 3 \begin{pmatrix} 1 \\ 0 \\ 0 \end{pmatrix} = \begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix}$$

Since $\|u'\|^2 = 6$, we have :

$$
\begin{align*}
Q_1 &:= H_{u'} \\
&= I_n - 2 \frac{u' u'^T}{\|u'\|^2} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} - \frac{2}{6} \begin{pmatrix} -1 \\ -2 \\ 1 \end{pmatrix} \begin{pmatrix} -1 & -2 & 1 \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} - \frac{1}{3} \begin{pmatrix} 1 & 2 & -1 \\ 2 & 4 & -2 \\ -1 & -2 & 1 \end{pmatrix} \\
&= \frac{1}{3} \begin{pmatrix} 2 & -2 & 1 \\ -2 & -1 & 2 \\ 1 & 2 & 2 \end{pmatrix}
\end{align*}
$$
We have :

$$
\begin{align*}
AQ_1 &= \frac{1}{3} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ -2 & -1 & 2 \\ 1 & 2 & 2 \end{pmatrix} \\
&= \begin{pmatrix} 3 & 0 & 0 \\ 4 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix}
\end{align*}
$$

Since $AQ_1$ is a lower triangular matrix, the algorithm is complete. We then set :

$Q := Q_1, \quad R := AQ.$

We indeed have $QQ^T = I_3$ i.e. $Q \in O_3(\mathbb{R})$. We have

$$
R := Q^T A = \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix} = \begin{pmatrix} 3 & -4 & 2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}.
$$

We can easily compute the inverse of $R$ using one of the methods previously discussed since $R$ is an upper triangular
matrix. For example, using the cofactor method, we have

$$
\mathrm{Com}(R) = \begin{pmatrix} 1 & 0 & 0 \\ 4 & 1 & 0 \\ -2 & 0 & 1 \end{pmatrix}
$$

we get :

$$
R = \frac{1}{\det(R)} \mathrm{Com}(R)^T = \frac{1}{3} \begin{pmatrix} 1 & 4 & -2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}
$$

Thus, since $A = QR$, we have :

$$
\begin{align*}
A^{-1} &= R^{-1}Q^{-1} \\
&= R^{-1}Q^T \\
&= \frac{1}{3} \begin{pmatrix} 1 & 4 & -2 \\ 0 & 3 & 0 \\ 0 & 0 & 3 \end{pmatrix} \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \\
&= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix}
\end{align*}
$$
# 15 Polar Decomposition

For any positive definite symmetric matrix, we can define the notion of a square root. This gives us the polar decomposition of the matrix.

By taking the product of an invertible matrix with its transpose (see also section 10), we construct a positive definite symmetric matrix. Using the polar decomposition of this matrix, we obtain another method to compute the inverse of a matrix.

## 15.1 Introduction

We start with the notion of the square root associated with a positive definite symmetric matrix.

**Lemma 15.1.1**

Let $C \in \text{GL}_n(\mathbb{R})$ be positive definite symmetric. Then there exists a unique positive definite symmetric matrix $R$ such that:

$$R^2 = C.$$

We denote it as:

$$\sqrt{C} := R$$

referred to as the **square root of C**.

**Proof.**

Let's show the existence of $R$. Let $\lambda_1, \dots, \lambda_n$ be the strictly positive eigenvalues of $C$. There exists an orthogonal matrix $Q$ such that:

$$C = Q \begin{pmatrix} \lambda_1 & & (0) \\ & \ddots & \\ (0) & & \lambda_n \end{pmatrix} Q^T.$$

Define:

$$R := Q \begin{pmatrix} \sqrt{\lambda_1} & & (0) \\ & \ddots & \\ (0) & & \sqrt{\lambda_n} \end{pmatrix} Q^T.$$

The matrix $R$ is symmetric because:

$$\begin{align*} R^T &= \left( Q \begin{pmatrix} \sqrt{\lambda_1} & & (0) \\ & \ddots & \\ (0) & & \sqrt{\lambda_n} \end{pmatrix} Q^T \right)^T \\ &= Q \begin{pmatrix} \sqrt{\lambda_1} & & (0) \\ & \ddots & \\ (0) & & \sqrt{\lambda_n} \end{pmatrix} Q^T \\ &= R \end{align*}$$

and $R$ is positive definite because its eigenvalues $\sqrt{\lambda_1}, \dots, \sqrt{\lambda_n}$ are strictly positive.

Let's show the uniqueness of $R$. Let $R$ be a positive definite symmetric matrix such that $R^2 = C$. Let $\mu_1, \dots, \mu_p$ be the distinct eigenvalues of $C$ ($p \le n$) and for each $i \in \{1, \dots, p\}$ let $n_i := \dim \mathbb{R} E_{\mu_i}(C)$. We have the direct sum (see subsection 8.1):

$$\mathbb{R}^n = \bigoplus_{i=1}^{p} E_{\mu_i}(C)$$
and let $\mathcal{B}$ be a basis adapted to this direct sum. Define $P$ as the change of basis matrix from the canonical basis of $\mathbb{R}^n$ to $\mathcal{B}$.

For each $i \in \{1, \dots, p\}$, the eigenspace $E_{\mu_i}(C)$ is stable under $R$ because for all $x \in E_{\mu_i}(C)$:

$$ 
\begin{aligned}
C(Rx) &= R^3 x \\
&= RCx \\
&= \mu_i Rx
\end{aligned}
$$

i.e., $Rx \in E_{\mu_i}(C)$. Thus, the matrix $S := P^{-1}RP$ is a block diagonal matrix of the form:

$$ S = P^{-1}RP = \begin{pmatrix} A_1 & & (0) \\ & \ddots & \\ (0) & & A_p \end{pmatrix} $$

where for each $i \in \{1, \dots, p\}$:

$$ A_i \in \mathrm{GL}_{n_i}(\mathbb{R}). $$

Thus, we have:

$$ 
\begin{aligned}
\begin{pmatrix} A_1^2 & & (0) \\ & \ddots & \\ (0) & & A_p^2 \end{pmatrix} &= S^2 \\
&= (P^{-1}RP)^2 \\
&= P^{-1}R^2P \\
&= P^{-1}CP \\
&= \begin{pmatrix} \mu_1 I_{n_1} & & (0) \\ & \ddots & \\ (0) & & \mu_p I_{n_p} \end{pmatrix}
\end{aligned}
$$

i.e., for each $i \in \{1, \dots, p\}$:

$$ A_i^2 = \mu_i I_{n_i}. $$

Moreover, for each $i \in \{1, \dots, p\}$, $A_i$ is symmetric and positive definite because as $R$ is symmetric, we have:

$$ 
\begin{aligned}
\begin{pmatrix} A_1^T & & (0) \\ & \ddots & \\ (0) & & A_p^T \end{pmatrix}^T &= \begin{pmatrix} A_1 & & (0) \\ & \ddots & \\ (0) & & A_p \end{pmatrix}^T \\
&= S^T \\
&= (P^{-1}RP)^T \\
&= P^{-1}R^T P \\
&= P^{-1}RP \\
&= \begin{pmatrix} A_1 & & (0) \\ & \ddots & \\ (0) & & A_p \end{pmatrix}
\end{aligned}
$$

i.e., for each $i \in \{1, \dots, p\}$, $A_i^T = A_i$, and since $R$ is positive definite, we have:

$$ 
\begin{aligned}
\chi_R(X) &= \det(R - XI_n) \\
&= \det(P^{-1}RP - XI_n)
\end{aligned}
$$
$$
\begin{align*}
&= \det(S - XI_n) \\
&= \begin{vmatrix}
A_1 - XI_{n_1} & & (0) \\
& (0) & A_p - XI_{n_p}
\end{vmatrix} \\
&= \prod_{i=1}^{p} \det(A_i - XI_n) \\
&= \prod_{i=1}^{p} \chi_{A_i}(X)
\end{align*}
$$

so for each $i \in \{1, \dots, p\}$ :

$$
\mathrm{Spec}(A_i) \subset \mathrm{Spec}(R) \subset \mathbb{R}_*^+.
$$

Let $i \in \{1, \dots, p\}$ and $\alpha$ be an eigenvalue (real) of $A_i$. For all $x \in E_\alpha(A_i)$ :

$$
\begin{align*}
\mu_i x &= \mu_i I_{n_i} \\
&= A_i^2 x \\
&= A_i \alpha x \\
&= \alpha^2 x
\end{align*}
$$

i.e., $\alpha^2 = \mu_i$ ($x$ is non-zero). Since $\alpha > 0$ (because $A_i$ is positive definite), we have $\alpha = \sqrt{\mu_i}$. Thus, $A_i$ is diagonalizable (because it is symmetric) and has a unique eigenvalue $\sqrt{\mu_i}$. Hence, there exists an invertible matrix $P'$ such that :

$$
(P')^{-1} A_i P' = \sqrt{\mu_i} I_{n_i}
$$

i.e., we have :

$$
\begin{align*}
A_i &= P' \sqrt{\mu_i} I_{n_i} (P')^{-1} \\
    &= \sqrt{\mu_i} I_{n_i}
\end{align*}
$$

(a matrix with only one eigenvalue is a homothety, thus $A_i$ is uniquely determined). We have thus shown that there is at most one matrix $R$ given by :

$$
\begin{align*}
R &= PSP^{-1} \\
  &= P \begin{pmatrix} A_1 & & (0) \\ & \ddots & A_p \\ (0) & & \end{pmatrix} P^{-1} \\
  &= P \begin{pmatrix} \sqrt{\mu_1} I_{n_1} & & (0) \\ & \ddots & \sqrt{\mu_p} I_{n_p} \\ (0) & & \end{pmatrix} P^{-1}.
\end{align*}
$$

Thus, the existence and uniqueness of the matrix $R$ are established.
We deduce the polar decomposition.
□

**Proposition 15.1.2: Polar Decomposition**

Let $A \in \text{GL}_n(\mathbb{R})$. Then there exists an orthogonal matrix $U \in \text{O}_n(\mathbb{R})$ and a positive definite symmetric matrix $P \in \text{GL}_n(\mathbb{R})$ such that :

$A = PU.$
**Proof.** Let's analyze to find the expressions for $U$ and $P$.

**Analysis.** Suppose $A = PU$ with $U$ being an orthogonal matrix and $P$ being a positive definite symmetric matrix. Since $P$ is positive definite, $P$ can be decomposed as :

$$P = QDQ^T$$

where $Q$ is orthogonal. Therefore, we have :

$$ 
\begin{aligned}
AA^T &= PU(PU)^T \\
&= PUU^T P^T \\
&= PP^T \\
&= P^2
\end{aligned}
$$

**Synthesis.** Since $AA^T$ is positive definite, we set (see the previous lemma 15.1.1) :

$$P := \sqrt{AA^T}, \quad U := P^{-1}A = (\sqrt{A^T A})^{-1}A.$$

The matrix $P$ is positive definite symmetric by the lemma and :

$$ 
\begin{aligned}
UU^T &= P^{-1}A(P^{-1}A)^T \\
&= P^{-1}AA^T(P^{-1})^T \\
&= P^{-1}P^2(P^{-1})^T \\
&= P^{-1}P(P^{-1}P)^T \\
&= I_n
\end{aligned}
$$

Thus $U$ is orthogonal, $P$ is positive definite symmetric, and $A = PU$.

We then obtain that :

$$ 
\begin{aligned}
A^{-1} &= A^T C^{-1} \\
&= A^T P D^{-1} P^T
\end{aligned}
$$

## 15.2 Example

Calculate the inverse of :

$$A := \begin{pmatrix} 2 & -2 & 1 \\ 2 & -3 & 2 \\ -1 & 2 & 0 \end{pmatrix}$$

using the polar decomposition. In section 10, we studied the matrix :

$$C := AA^T.$$

We found that $C$ is diagonalizable (since it is symmetric) and :

$$ 
\begin{aligned}
C &= AA^T = \begin{pmatrix} 9 & 12 & -6 \\ 12 & 17 & -8 \\ -6 & -8 & 5 \end{pmatrix} \\
\chi_C(X) &= 6(1-X)(X^2 - 30X + 9)
\end{aligned}
$$
$$ \mathrm{Sp}(C) = \{1, 15 \pm 6\sqrt{6}\} = \{1, \lambda_1, \lambda_{-1}\} $$

$$ E_1(C) = \mathrm{Vect} \begin{pmatrix} 0 \\ 1 \\ 2 \end{pmatrix} = \mathrm{Vect} \begin{pmatrix} 1 \\ \sqrt{5} \\ 1 \end{pmatrix} $$

$$ E_{\lambda_e}(C) = \mathrm{Vect} \begin{pmatrix} 1 - e\sqrt{6} \\ -2 \\ 1 \end{pmatrix} = \mathrm{Vect} \begin{pmatrix} \alpha_e \begin{pmatrix} 1 - e\sqrt{6} \\ -2 \\ 1 \end{pmatrix} \end{pmatrix} $$

with $e \in \{\pm 1\}$ :

$$ \alpha_e := \frac{1}{\sqrt{12 - 2e\sqrt{6}}} $$

Thus the orthogonal change of basis matrix is :

$$ P := \begin{pmatrix} 0 & \alpha_1 (1 - \sqrt{6}) & \alpha_{-1} (1 + \sqrt{6}) \\ \frac{1}{\sqrt{5}} & -2\alpha_1 & -2\alpha_{-1} \\ \frac{2}{\sqrt{5}} & \alpha_1 & \alpha_{-1} \end{pmatrix}. $$

Let :

$$ D := \begin{pmatrix} 1 & 0 & 0 \\ 0 & 15 + 6\sqrt{6} & 0 \\ 0 & 0 & 15 - 6\sqrt{6} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & \lambda_1 & 0 \\ 0 & 0 & \lambda_{-1} \end{pmatrix} $$

So we have :

$$ \begin{align*} C^{-1} &= PD^{-1}P^T \\ &= \begin{pmatrix} 0 & \alpha_1 (1 - \sqrt{6}) & \alpha_{-1} (1 + \sqrt{6}) \\ \frac{1}{\sqrt{5}} & -2\alpha_1 & -2\alpha_{-1} \\ \frac{2}{\sqrt{5}} & \alpha_1 & \alpha_{-1} \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & \frac{1}{\lambda_1} & 0 \\ 0 & 0 & \frac{1}{\lambda_{-1}} \end{pmatrix} \begin{pmatrix} 0 & \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}} \\ \alpha_1 (1 - \sqrt{6}) & -2\alpha_1 & \alpha_1 \\ \alpha_{-1} (1 + \sqrt{6}) & -2\alpha_{-1} & \alpha_{-1} \end{pmatrix} \\ &= \begin{pmatrix} 0 & \alpha_1 (1 - \sqrt{6}) & \alpha_{-1} (1 + \sqrt{6}) \\ \frac{1}{\sqrt{5}} & -2\alpha_1 & -2\alpha_{-1} \\ \frac{2}{\sqrt{5}} & \alpha_1 & \alpha_{-1} \end{pmatrix} \begin{pmatrix} 0 & \frac{1}{\sqrt{5}} & \frac{2}{\sqrt{5}} \\ \alpha_1 (1 - \sqrt{6}) / \lambda_1 & -2\alpha_1 / \lambda_1 & \alpha_1 / \lambda_1 \\ \alpha_{-1} (1 + \sqrt{6}) / \lambda_{-1} & -2\alpha_{-1} / \lambda_{-1} & \alpha_{-1} / \lambda_{-1} \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 7 & -4 & 2 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} \end{align*} $$

Since $C^{-1} = (A^T)^{-1}A^{-1}$, we have :

$$ \begin{align*} A^{-1} &= A^T C^{-1} \\ &= \frac{1}{3} \begin{pmatrix} 2 & 2 & -1 \\ -2 & -3 & 2 \\ 1 & 2 & 0 \end{pmatrix} \begin{pmatrix} 7 & -4 & 2 \\ -4 & 3 & 0 \\ 2 & 0 & 3 \end{pmatrix} \\ &= \frac{1}{3} \begin{pmatrix} 4 & -2 & 1 \\ 2 & -1 & 2 \\ -1 & 2 & 2 \end{pmatrix} \end{align*} $$
# A Operation Matrices

We define the three elementary operation matrices. To simplify the definitions, we use the following notations.

* The Kronecker symbol is defined for all elements $a, b$ by :

$$ \delta_{a,b} := \begin{cases} 1 & \text{if } a = b \\ 0 & \text{otherwise.} \end{cases} $$

* For any matrix $M \in \mathcal{M}_{n,p}(\mathbb{R})$, any $k \in \{1, \dots, n\}$, and any $l \in \{1, \dots, p\}$, we denote $[M]_{k,l}$ as the coefficient of $M$ in row $k$ and column $l$.

* We will omit the comma in the subscript to separate indices if no confusion is possible. For example, we will write $a_{ij} := a_{i,j}$ but leave $a_{i+2,j}$.

* We denote $\langle \cdot, \cdot \rangle$ as the usual inner product on $\mathbb{R}^n$ and $||.||$ as the associated Euclidean norm. For all $u, v \in \mathbb{R}^n$ we have :

$$ \langle u, v \rangle = u^T v \\ \langle v, v \rangle = v^T v = ||v||^2 $$

## A.1 Symmetric Groups

### Definition A.1.1

(i) A **permutation of** $\{1, \dots, n\}$ is a bijection $\sigma$ from $\{1, \dots, n\}$ to $\{1, \dots, n\}$. Its support is defined by :

$$ \text{Supp}(\sigma) := \{i \in \{1, \dots, n\}, \sigma(i) \neq i\}. $$

We denote $\mathfrak{S}_n$ as the set of permutations of the set $\{1, \dots, n\}$.

(ii) Let $k \in \{2, \dots, n\}$ and $i_1, \dots, i_k$ be distinct elements of $\{1, \dots, n\}$. The **cycle** $\sigma := (i_1, \dots, i_k)$ **of length** $k$ is the permutation of $\{1, \dots, n\}$ such that :

$$ \forall j \in \{1, \dots, k-1\}, \sigma(i_j) = i_{j+1} \\ \sigma(i_k) = i_1 \\ \text{Supp}(\sigma) = \{i_1, \dots, i_k\} $$

(iii) A **transposition** $\tau$ **of** $\{1, \dots, n\}$ is a cycle of length 2, i.e., there exist $i, j \in \{1, \dots, n\}$ distinct such that $\text{Supp}(\tau) = \{i, j\}$. We denote it as :

$$ \tau_{i,j} := \tau = (i,j). $$

(iv) For any permutation $\sigma$ of $\{1, \dots, n\}$, the **signature of** $\sigma$ is defined by :

$$ \varepsilon(\sigma) := \prod_{1 \le i < j \le n} \frac{\sigma(j) - \sigma(i)}{j - i}. $$

Composition is an internal operation in $\mathfrak{S}_n$ that makes $\mathfrak{S}_n$ a group and :

$$ \text{Card}(\mathfrak{S}_n) = n! $$

A permutation $\sigma$ can be noted in two ways :
• In matrix form :

$$
\sigma = \begin{pmatrix} 1 & 2 & \cdots & n \\ \sigma(1) & \sigma(2) & \cdots & \sigma(n) \end{pmatrix}
$$

• In tabular form :

<table>
  <tr>
    <td><i>i</i></td>
    <td>1</td>
    <td>2</td>
    <td>...</td>
    <td><i>n</i></td>
  </tr>
  <tr>
    <td><i>σ</i>(<i>i</i>)</td>
    <td><i>σ</i>(1)</td>
    <td><i>σ</i>(2)</td>
    <td>...</td>
    <td><i>σ</i>(<i>n</i>)</td>
  </tr>
</table>

The following proposition states the results useful for understanding the text.

**Proposition A.1.2**

(i) Every permutation $\sigma$ can be written as a product of transpositions.

(ii) The signature defines a group homomorphism :

$$
\varepsilon: \left( \mathfrak{S}_n, \circ \right) \longrightarrow \left( \{\pm 1\}, \times \right) \\
\sigma \longmapsto \varepsilon(\sigma)
$$

**Proof.** We provide an outline of the proofs.

(i) This result is proved in two steps. Suppose $\sigma$ is different from the identity, i.e., its support is non-empty.

(a) First, every permutation $\sigma$ can be written as a product of cycles. To see this, let :

$$
\{i_1, \dots, i_k\} := \mathrm{Supp}(\sigma).
$$

Take the first element, $i_{j_1} := i_1$ and construct a cycle by successively setting :

$$
i_{j_1} := i_1, \quad i_{j_2} := \sigma(i_{j_1}), \quad i_{j_3} := \sigma(i_{j_2}), \dots
$$

Since the support is finite, by the pigeonhole principle and as $\sigma$ is bijective, there exists a first integer $r \ge 2$ such that :

$$
i_{j_1} := \sigma(i_{j_r}).
$$

We have then defined the cycle of length $r$ :

$$
c_1 := (i_{j_1}, \dots, i_{j_r})
$$

Then take an element from (if this set is non-empty, otherwise stop) :

$$
\mathrm{Supp}(\sigma) - \{i_{j_1}, \dots, i_{j_r}\}
$$

and similarly construct a cycle $c_2$ starting with this element. Repeat this process until all elements of $\mathrm{Supp}(\sigma)$ appear in one of the cycles. There exist then $s \in \mathbb{N}_*$ and cycles $c_1, \dots, c_s$ such that :

$$
\sigma = c_1 \circ \cdots \circ c_s.
$$

(b) Second, every cycle can be written as a product of transpositions. Let a cycle $(i_1, \dots, i_k)$ in $\{1, \dots, n\}$.
We have the simple formula :

$$
(i_1, \dots, i_k) = (i_1, i_2) \circ (i_2, i_3) \circ \dots \circ (i_{k-1}, i_k).
$$
(ii) For any permutation $\sigma \in \mathfrak{S}_n$:

$$ \varepsilon(\sigma) = \prod_{1 \le i < j \le n} \frac{\sigma(j) - \sigma(i)}{j - i} = \prod_{\{i,j\} \in \mathcal{P}} \frac{\sigma(j) - \sigma(i)}{j - i} $$

with:

$$ \mathcal{P} := \{\{i, j\}, 1 \le i \le n \land 1 \le j \le n \land i \neq j\}. $$

Let $\sigma, \tau \in \mathfrak{S}_n$. We show that:

$$ \varepsilon(\sigma \circ \tau) = \varepsilon(\sigma)\varepsilon(\tau). $$

We have:

$$ \begin{aligned} \varepsilon(\sigma \circ \tau) &= \prod_{\{i,j\} \in \mathcal{P}} \frac{(\sigma \circ \tau)(j) - (\sigma \circ \tau)(i)}{j - i} \\ &= \prod_{\{i,j\} \in \mathcal{P}} \frac{\sigma(\tau(j)) - \sigma(\tau(i))}{\tau(j) - \tau(i)} \prod_{\{i,j\} \in \mathcal{P}} \frac{\tau(j) - \tau(i)}{j - i} \\ &= \prod_{\{k,l\} \in \mathcal{P}} \frac{\sigma(k) - \sigma(l)}{k - l} \varepsilon(\tau) \\ &= \varepsilon(\sigma)\varepsilon(\tau) \end{aligned} $$

<figure><img src="image_11.png" /></figure>

We can also show that the parity of the number of transpositions appearing in the decomposition does not depend on the decomposition (i.e. if we have two decompositions of $\sigma$ into a product of transpositions with $N$ and $N'$ transpositions respectively, then $N' \equiv N[2]$). We then have:

$$ \varepsilon(\sigma) = (-1)^N. $$

Let $c := (i_1, \dots, i_k)$ be a cycle of $\{1, \dots, n\}$. We have:

$$ \varepsilon(c) = (-1)^k $$

and in particular, the signature of a transposition is $-1$ (cycle of length 2). We provide two simple examples to understand the defined objects and notions.

### Example A.1.3

(1) The symmetric group $\mathfrak{S}_3$ has order $6 = 3!$, and we have:

$$ \begin{aligned} \mathfrak{S}_3 &= \left\{ \begin{pmatrix} 1 & 2 & 3 \\ 1 & 2 & 3 \\ 2 & 1 & 3 \end{pmatrix}, \begin{pmatrix} 1 & 2 & 3 \\ 2 & 1 & 3 \\ 3 & 2 & 1 \end{pmatrix}, \begin{pmatrix} 1 & 2 & 3 \\ 3 & 2 & 1 \\ 1 & 3 & 2 \end{pmatrix}, \begin{pmatrix} 1 & 2 & 3 \\ 2 & 3 & 1 \\ 1 & 3 & 2 \end{pmatrix}, \begin{pmatrix} 1 & 2 & 3 \\ 3 & 1 & 2 \\ 2 & 1 & 3 \end{pmatrix} \right\} \\ &= \{\mathrm{Id}, (12), (13), (23), (123) = (12)(23), (132) = (13)(32)\} \\ &=: \{\sigma_1, \sigma_2, \sigma_3, \sigma_4, \sigma_5, \sigma_6\} \end{aligned} $$

Since the signature of a transposition is $-1$, the signatures of the six permutations are:

$$ \begin{aligned} \varepsilon(\sigma_1) &= 1 \\ \varepsilon(\sigma_4) &= -1 \end{aligned} $$

$$ \begin{aligned} \varepsilon(\sigma_2) &= -1 \\ \varepsilon(\sigma_5) &= \varepsilon(\sigma_2)\varepsilon(\sigma_4) = 1 \end{aligned} $$

$$ \begin{aligned} \varepsilon(\sigma_3) &= -1 \\ \varepsilon(\sigma_6) &= \varepsilon(\sigma_3)\varepsilon(\sigma_4) = 1 \end{aligned} $$
Let us show using the other formula that $\varepsilon(\sigma_6) = 1$. Since :

$$ \sigma_6 = \begin{pmatrix} 1 & 2 & 3 \\ 3 & 1 & 2 \end{pmatrix} $$

we have :

$$ \begin{aligned} \varepsilon(\sigma_6) &= \prod_{1 \le i < j \le 3} \frac{\sigma_6(j) - \sigma_6(i)}{j - i} \\ &= \frac{\sigma_6(2) - \sigma_6(1)}{2 - 1} \cdot \frac{\sigma_6(3) - \sigma_6(1)}{3 - 1} \cdot \frac{\sigma_6(3) - \sigma_6(2)}{3 - 2} \\ &= \frac{1 - 3}{2 - 1} \cdot \frac{2 - 3}{3 - 1} \cdot \frac{2 - 1}{3 - 2} \\ &= -1 \times -1 \times 1 \\ &= 1 \end{aligned} $$

(2) Consider the permutation of $\{1, \dots, 12\}$ :

$$ \sigma := \begin{pmatrix} 1 & 2 & 3 & 4 & 5 & 6 & 7 & 8 & 9 & 10 & 11 & 12 \\ 2 & 4 & 3 & 10 & 8 & 9 & 7 & 12 & 6 & 1 & 11 & 5 \end{pmatrix} $$

We have :

$$ \text{Supp}(\sigma) = \{1, 2, 4, 5, 6, 8, 9, 10, 12\}. $$

Construct, as in the proof, a decomposition into a product of transpositions for $\sigma$.

* Take the first element of the support 1 and construct the first cycle with elements :

$$ 1, \sigma(1) = 2, \sigma(2) = 4, \sigma(4) = 10, \sigma(10) = 1. $$

We then have a cycle of length $4 c_1 := (1, 2, 4, 10)$.

* Since there are remaining elements not chosen, take the first unchosen element 5 and construct the second cycle with elements :

$$ 5, \sigma(5) = 8, \sigma(8) = 12, \sigma(12) = 5. $$

We then have a cycle of length $3 c_2 := (5, 8, 12)$.

* Since there are still elements not chosen, take the first unchosen element 6 and construct the third cycle with elements :

$$ 6, \sigma(6) = 9. $$

We then have a cycle of length $2 c_3 := (6, 9)$.

* There are no more elements in the support that do not appear in a cycle. We have :

$$ \sigma = c_1 \circ c_2 \circ c_3 = (1, 2, 4, 10) \circ (5, 8, 12) \circ (6, 9). $$

* Then write each cycle as a product of transpositions :

$$ c_1 = (1, 2, 4, 10) = (1, 2) \circ (2, 4) \circ (4, 10) $$

$$ c_2 = (5, 8, 12) = (5, 8) \circ (8, 12) $$

$$ c_3 = (6, 9) $$

Thus we have :

$$ \sigma = (1, 2) \circ (2, 4) \circ (4, 10) \circ (5, 8) \circ (8, 12) \circ (6, 9) $$

and we have :

$$ \varepsilon(\sigma) = (-1)^6 = 1 $$
## A.2 Determinant of a Matrix

This subsection provides only the essential concepts about determinants to understand the text. Let $(e_1, \dots, e_n)$ be the canonical basis of $\mathbb{R}^n$ defined by :

$$e_1 := \begin{pmatrix} 1 \\ 0 \\ 0 \\ \vdots \\ 0 \end{pmatrix}, \quad e_2 := \begin{pmatrix} 0 \\ 1 \\ 0 \\ \vdots \\ 0 \end{pmatrix}, \dots, e_n := \begin{pmatrix} 0 \\ 0 \\ \vdots \\ 0 \\ 1 \end{pmatrix}.$$

### Definition A.2.1

Let $f$ be a mapping from $(\mathbb{R}^n)^n$ to $\mathbb{R}$. We say that $f$ is an $n$-linear alternating form if :

(i) (linearity in each variable) for all $i \in \{1, \dots, n\}$, all $x_1, \dots, x_n, y_i \in \mathbb{R}^n$ and all $\alpha, \beta \in \mathbb{R}$, we have :

$$f(x_1, \dots, x_{i-1}, \alpha x_i + \beta y_i, x_{i+1}, \dots, x_n) = \alpha f(x_1, \dots, x_n) + \beta f(x_1, \dots, y_i, \dots, x_n);$$

(ii) (alternating) for all distinct $i, j \in \{1, \dots, n\}$ and all $x_1, \dots, x_n \in \mathbb{R}^n$ such that $x_i = x_j$, we have :

$$f(x_1, \dots, x_n) = 0.$$

### Proposition A.2.2

Let $f$ be an $n$-linear alternating form. Then we have :

$$f(x_1, \dots, x_n) = \left( \sum_{\sigma \in \mathcal{S}_n} \varepsilon(\sigma) \prod_{j=1}^{n} x_{\sigma(j),j} \right) f(e_1, \dots, e_n).$$

**Proof.** We will show the result for the case $n := 3$ (we say that $f$ is trilinear alternating). Expanding successively by linearity with respect to each variable (we put 'alt' under the equal sign to signify that we use alternation and 'lin' to signify that we use linearity with respect to one variable) :

$$\begin{align*}
f(x_1, x_2, x_3) &= f(x_{11}e_1 + x_{21}e_2 + x_{31}e_3, x_2, x_3) \\
\operatorname{lin} &= x_{11}f(e_1, x_2, x_3) + x_{21}f(e_2, x_2, x_3) + x_{31}f(e_3, x_2, x_3) \\
&= x_{11}f(e_1, x_{12}e_1 + x_{22}e_2 + x_{32}e_3, x_3) \\
&\quad + x_{21}f(e_2, x_{12}e_1 + x_{22}e_2 + x_{32}e_3, x_3) \\
&\quad + x_{31}f(e_3, x_{12}e_1 + x_{22}e_2 + x_{32}e_3, x_3) \\
\operatorname{lin} &= x_{11}[x_{12}f(e_1, e_1, x_3) + x_{22}f(e_1, e_2, x_3) + x_{32}f(e_1, e_3, x_3)] \\
&\quad + x_{21}[x_{12}f(e_2, e_1, x_3) + x_{22}f(e_2, e_2, x_3) + x_{32}f(e_2, e_3, x_3)] \\
&\quad + x_{31}[x_{12}f(e_3, e_1, x_3) + x_{22}f(e_3, e_2, x_3) + x_{32}f(e_3, e_3, x_3)] \\
\operatorname{alt} &= x_{11}[0 + x_{22}f(e_1, e_2, x_3) + (-1)^1 x_{32}f(e_1, x_3, e_3)] \\
&\quad + x_{21}[(-1)^1 x_{12}f(e_1, e_2, x_3) + 0 + (-1)^2 x_{32}f(x_3, e_2, e_3)] \\
&\quad + x_{31}[(-1)^2 x_{12}f(e_1, x_3, e_3) + (-1)^1 x_{22}f(x_3, e_2, e_3) + 0] \\
&= (x_{11}x_{22} + (-1)^1 x_{21}x_{12}) f(e_1, e_2, x_{13}e_1 + x_{23}e_2 + x_{33}e_3) \\
&\quad + ((-1)^1 x_{11}x_{32} + (-1)^2 x_{31}x_{12}) f(e_1, x_{13}e_1 + x_{23}e_2 + x_{33}e_3, e_3) \\
&\quad + ((-1)^1 x_{31}x_{22} + (-1)^2 x_{21}x_{32}) f(x_{13}e_1 + x_{23}e_2 + x_{33}e_3, e_2, e_3)
\end{align*}$$
$$
\begin{align*}
\text{lin} &= (x_{11}x_{22} + (-1)^1x_{21}x_{12}) (x_{13}f(e_1, e_2, e_1) + x_{23}f(e_1, e_2, e_2) + x_{33}f(e_1, e_2, e_3)) \\
&\quad + ((-1)^1x_{11}x_{32} + (-1)^2x_{31}x_{12}) (x_{13}f(e_1, e_1, e_3) + x_{23}f(e_1, e_2, e_3) + x_{33}f(e_1, e_3, e_3)) \\
&\quad + ((-1)^1x_{31}x_{22} + (-1)^2x_{21}x_{32}) (x_{13}f(e_1, e_2, e_3) + x_{23}f(e_2, e_2, e_3) + x_{33}f(e_3, e_2, e_3)) \\
\text{alt} &= (x_{11}x_{22} + (-1)^1x_{21}x_{12}) (x_{33}f(e_1, e_2, e_1) + 0 + 0) \\
&\quad + ((-1)^1x_{11}x_{32} + (-1)^2x_{31}x_{12}) (0 + x_{23}f(e_1, e_2, e_3) + 0) \\
&\quad + ((-1)^1x_{31}x_{22} + (-1)^2x_{21}x_{32}) (x_{13}f(e_1, e_2, e_3) + 0 + 0) \\
&= (x_{11}x_{22} + (-1)^1x_{21}x_{12}) x_{33}f(e_1, e_2, e_1) \\
&\quad + ((-1)^1x_{11}x_{32} + (-1)^2x_{31}x_{12}) x_{23}f(e_1, e_2, e_3) \\
&\quad + ((-1)^1x_{31}x_{22} + (-1)^2x_{21}x_{32}) x_{13}f(e_1, e_2, e_3)
\end{align*}
$$

We have :

$$
\begin{align*}
\mathfrak{S}_3 &= \{\mathrm{Id}, (12), (13), (23), (123) = (12)(23), (132) = (13)(32)\} \\
&=: \{\sigma_1, \sigma_2, \sigma_3, \sigma_4, \sigma_5, \sigma_6\}
\end{align*}
$$

and the signatures of the six permutations are :

$$
\varepsilon(\sigma_1) = (-1)^0 = 1
$$

$$
\varepsilon(\sigma_2) = (-1)^1 = -1
$$

$$
\varepsilon(\sigma_3) = (-1)^1 = -1
$$

$$
\varepsilon(\sigma_4) = (-1)^1 = -1
$$

$$
\varepsilon(\sigma_5)(-1)^2 = 1
$$

$$
\varepsilon(\sigma_6) = (-1)^2 = 1
$$

So we have :

$$
\begin{align*}
f(x_1, x_2, x_3) \\
&= [x_{11}x_{22}x_{33} + (-1)^1x_{12}x_{21}x_{33} + (-1)^1x_{11}x_{23}x_{32} \\
&\quad + (-1)^2x_{12}x_{23}x_{31} + (-1)^1x_{13}x_{22}x_{31} + (-1)^2x_{13}x_{21}x_{32}] f(e_1, e_2, e_3) \\
&= \left[ \varepsilon(\sigma_1) \prod_{i=1}^3 x_{i,\sigma_1(i)} + \varepsilon(\sigma_2) \prod_{i=1}^3 x_{i,\sigma_2(i)} + \varepsilon(\sigma_4) \prod_{i=1}^3 x_{i,\sigma_4(i)} \right. \\
&\qquad \left. \varepsilon(\sigma_5) \prod_{i=1}^3 x_{i,\sigma_5(i)} + \varepsilon(\sigma_3) \prod_{i=1}^3 x_{i,\sigma_3(i)} + \varepsilon(\sigma_6) \prod_{i=1}^3 x_{i,\sigma_6(i)} \right] f(e_1, e_2, e_3) \\
&= \left[ \sum_{k=1}^6 \varepsilon(\sigma_k) \prod_{i=1}^3 x_{i,\sigma_k(i)} \right] f(e_1, e_2, e_3) \\
&= \left[ \sum_{\sigma \in \mathfrak{S}_3} \varepsilon(\sigma) \prod_{i=1}^n x_{i,\sigma(i)} \right] f(e_1, e_2, e_3)
\end{align*}
$$

The set of *n*-linear alternating forms forms a vector space of dimension 1. It is therefore spanned by any non-
zero *n*-linear alternating form. Hence the following definitions.

**Definition A.2.3**

(i) The **determinant** det is the unique *n*-linear alternating form such that :

$$
\det(e_1, \dots, e_n) = 1.
$$
(ii) Let the matrix defined by its columns be :

$$
M := (C_1 | \cdots | C_n) \in \mathcal{M}_n(\mathbb{R}).
$$

The **determinant** of M is defined by :

$$
\det(M) := \det(C_1, \dots, C_n).
$$

For $M = (m_{i,j})_{ij}$, we have :

$$
\det(M) = \sum_{\sigma \in \mathfrak{S}_n} \varepsilon(\sigma) \prod_{i=1}^{n} m_{i, \sigma(i)}.
$$

We have the following useful properties of determinants.

**Proposition A.2.4**

Let $M, N \in \mathcal{M}_n(\mathbb{R})$.

(i) We have :

$$
\det(M^T) = \det(M).
$$

(ii) We have :

$$
\det(MN) = \det(M) \det(N).
$$

(iii) We have :

$$
\det(M) \neq 0 \iff M \in \mathrm{GL}_n(\mathbb{R}).
$$

A.3 Permutation Matrices

**Definition A.3.1**

Let $\sigma \in \mathfrak{S}_n$. The **permutation matrix associated with** $\sigma$ is the matrix $\mathbf{P}_{\sigma} \in \mathcal{M}_n(\mathbb{R})$ such that for all $k, l \in \{1, \dots, n\}$:

$$
[\mathbf{P}_{\sigma}]_{k,l} := \delta_{k,\sigma(l)}.
$$

We revisit example (1) from A.1.3.

**Example A.3.2**

We have :

$$
\begin{align*}
\mathfrak{S}_3 &= \{\mathrm{Id}, (12), (13), (23), (123) = (12)(23), (132) = (13)(32)\} \\
&=: \{\sigma_1, \sigma_2, \sigma_3, \sigma_4, \sigma_5, \sigma_6\}
\end{align*}
$$

The six associated permutation matrices are :

$$
\mathbf{P}_{\sigma_1} = \mathbf{P}_{\mathrm{Id}} = \begin{pmatrix} \delta_{1,\sigma_1(1)} & \delta_{1,\sigma_1(2)} & \delta_{1,\sigma_1(3)} \\ \delta_{2,\sigma_1(1)} & \delta_{2,\sigma_1(2)} & \delta_{2,\sigma_1(3)} \\ \delta_{3,\sigma_1(1)} & \delta_{3,\sigma_1(2)} & \delta_{3,\sigma_1(3)} \end{pmatrix} = \begin{pmatrix} \delta_{1,1} & \delta_{1,2} & \delta_{1,3} \\ \delta_{2,1} & \delta_{2,2} & \delta_{2,3} \\ \delta_{3,1} & \delta_{3,2} & \delta_{3,3} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}
$$
$$ \mathbf{P}_{\sigma_2} = \mathbf{P}_{(1,2)} = \begin{pmatrix} \delta_{1,\sigma_2(1)} & \delta_{1,\sigma_2(2)} & \delta_{1,\sigma_2(3)} \\ \delta_{2,\sigma_2(1)} & \delta_{2,\sigma_2(2)} & \delta_{2,\sigma_2(3)} \\ \delta_{3,\sigma_2(1)} & \delta_{3,\sigma_2(2)} & \delta_{3,\sigma_2(3)} \end{pmatrix} = \begin{pmatrix} \delta_{1,2} & \delta_{1,1} & \delta_{1,3} \\ \delta_{2,2} & \delta_{2,1} & \delta_{2,3} \\ \delta_{3,2} & \delta_{3,1} & \delta_{3,3} \end{pmatrix} = \begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{P}_{\sigma_3} = \mathbf{P}_{(1,3)} = \begin{pmatrix} \delta_{1,\sigma_3(1)} & \delta_{1,\sigma_3(2)} & \delta_{1,\sigma_3(3)} \\ \delta_{2,\sigma_3(1)} & \delta_{2,\sigma_3(2)} & \delta_{2,\sigma_3(3)} \\ \delta_{3,\sigma_3(1)} & \delta_{3,\sigma_3(2)} & \delta_{3,\sigma_3(3)} \end{pmatrix} = \begin{pmatrix} \delta_{1,3} & \delta_{1,2} & \delta_{1,1} \\ \delta_{2,3} & \delta_{2,2} & \delta_{2,1} \\ \delta_{3,3} & \delta_{3,2} & \delta_{3,1} \end{pmatrix} = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} $$

$$ \mathbf{P}_{\sigma_4} = \mathbf{P}_{(2,3)} = \begin{pmatrix} \delta_{1,\sigma_4(1)} & \delta_{1,\sigma_4(2)} & \delta_{1,\sigma_4(3)} \\ \delta_{2,\sigma_4(1)} & \delta_{2,\sigma_4(2)} & \delta_{2,\sigma_4(3)} \\ \delta_{3,\sigma_4(1)} & \delta_{3,\sigma_4(2)} & \delta_{3,\sigma_4(3)} \end{pmatrix} = \begin{pmatrix} \delta_{1,1} & \delta_{1,3} & \delta_{1,2} \\ \delta_{2,1} & \delta_{2,3} & \delta_{2,2} \\ \delta_{3,1} & \delta_{3,3} & \delta_{3,2} \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix} $$

$$ \mathbf{P}_{\sigma_5} = \mathbf{P}_{(1,2,3)} = \begin{pmatrix} \delta_{1,\sigma_1(1)} & \delta_{1,\sigma_1(2)} & \delta_{1,\sigma_1(3)} \\ \delta_{2,\sigma_1(1)} & \delta_{2,\sigma_1(2)} & \delta_{2,\sigma_1(3)} \\ \delta_{3,\sigma_1(1)} & \delta_{3,\sigma_1(2)} & \delta_{3,\sigma_1(3)} \end{pmatrix} = \begin{pmatrix} \delta_{1,2} & \delta_{1,3} & \delta_{1,1} \\ \delta_{2,2} & \delta_{2,3} & \delta_{2,1} \\ \delta_{3,2} & \delta_{3,3} & \delta_{3,1} \end{pmatrix} = \begin{pmatrix} 0 & 0 & 1 \\ 1 & 0 & 0 \\ 0 & 1 & 0 \end{pmatrix} $$

$$ \mathbf{P}_{\sigma_6} = \mathbf{P}_{(1,3,2)} = \begin{pmatrix} \delta_{1,\sigma_1(1)} & \delta_{1,\sigma_1(2)} & \delta_{1,\sigma_1(3)} \\ \delta_{2,\sigma_1(1)} & \delta_{2,\sigma_1(2)} & \delta_{2,\sigma_1(3)} \\ \delta_{3,\sigma_1(1)} & \delta_{3,\sigma_1(2)} & \delta_{3,\sigma_1(3)} \end{pmatrix} = \begin{pmatrix} \delta_{1,3} & \delta_{1,1} & \delta_{1,2} \\ \delta_{2,3} & \delta_{2,1} & \delta_{2,2} \\ \delta_{3,3} & \delta_{3,1} & \delta_{3,2} \end{pmatrix} = \begin{pmatrix} 0 & 1 & 0 \\ 0 & 0 & 1 \\ 1 & 0 & 0 \end{pmatrix} $$

We then have the following fundamental results.

### Lemma A.3.3

(i) Let $\sigma, \tau \in \mathfrak{S}_n$. We have :

$$ \mathbf{P}_{\sigma \circ \tau} = \mathbf{P}_{\sigma} \mathbf{P}_{\tau}. $$

(ii) Let $\sigma \in \mathfrak{S}_n$. We have $\mathbf{P}_{\sigma} \in \mathrm{GL}_n(\mathbb{R})$ and :

$$ \mathbf{P}_{\sigma}^{-1} = \mathbf{P}_{\sigma^{-1}} = \mathbf{P}_{\sigma}^{T}. $$

### Proof.

(i) Since $\tau$ is a bijection, for all $p, q \in \{1, \dots, n\}$ :

$$ p = \tau(q) \iff q = \tau^{-1}(p). $$

Thus for all $k, l \in \{1, \dots, n\}$, we have :

$$ \begin{align*} [\mathbf{P}_{\sigma}\mathbf{P}_{\tau}]_{kl} &= \sum_{m=1}^{n} [\mathbf{P}_{\sigma}]_{km} [\mathbf{P}_{\tau}]_{ml} \\ &= \sum_{m=1}^{n} \delta_{k,\sigma(m)} \delta_{m,\tau(l)} \\ &= \delta_{k,\sigma(\tau(l))} \\ &= [\mathbf{P}_{\sigma \circ \tau}]_{kl} \end{align*} $$

(ii) For all $k, l \in \{1, \dots, n\}$, we have :

$$ \begin{align*} [\mathbf{P}_{\sigma}\mathbf{P}_{\sigma}^{T}]_{kl} &= \sum_{m=1}^{n} [\mathbf{P}_{\sigma}]_{km} [\mathbf{P}_{\sigma}^{T}]_{ml} \\ &= \sum_{m=1}^{n} [\mathbf{P}_{\sigma}]_{km} [\mathbf{P}_{\sigma}]_{lm} \end{align*} $$
$$
\begin{align*}
&= \sum_{m=1}^{n} \delta_{k,\sigma(m)} \delta_{l,\sigma(m)} \\
&= \delta_{k,l} \\
&= [I_n]_{kl}
\end{align*}
$$

and by the bijection of $\sigma$, we have :

$$
\begin{align*}
[\mathbf{P}_{\sigma} \mathbf{P}_{\sigma^{-1}}]_{kl} &= \sum_{m=1}^{n} [\mathbf{P}_{\sigma}]_{km} [\mathbf{P}_{\sigma^{-1}}]_{ml} \\
&= \sum_{m=1}^{n} \delta_{k,\sigma(m)} \delta_{m,\sigma} \\
&= \sum_{m=1}^{n} \delta_{k,\sigma(\sigma^{-1}(l))} \\
&= \delta_{k,l} \\
&= [I_n]_{kl}
\end{align*}
$$

So we have $\mathbf{P}_{\sigma} \in \text{GL}_n(\mathbb{R})$ and :

$$
\mathbf{P}_{\sigma}^{-1} = \mathbf{P}_{\sigma^{-1}} = \mathbf{P}_{\sigma}^{T}.
$$

Thus the inverse of a permutation matrix is a permutation matrix.

A.4 Elementary Operation Matrices

**Definition A.4.1**

(i) Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$. The **permutation matrix** $(i, j)$ **of size** $n$ is the matrix :

$$
\mathbf{P}_{i,j} := \mathbf{P}_{\tau_{i,j}}
$$

where $\tau_{i,j}$ is the transposition of $i$ and $j$. It is also called an **elementary permutation matrix** or a **transposition matrix**.

(ii) Let $i \in \{1, \dots, n\}$ and $a \in \mathbb{R}^*$. The **dilation matrix** $(i, a)$ **of size** $n$ is the matrix $\mathbf{D}_i(a) \in \mathcal{M}_n(\mathbb{R})$ such that for all $k, l \in \{1, \dots, n\}$ :

$$
[\mathbf{D}_i(a)]_{k,l} := \begin{cases} a & \text{if } k = l = i \\ \delta_{k,l} & \text{otherwise.} \end{cases}
$$

(iii) Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$ and $b \in \mathbb{R}$. The **shear matrix** $(i, j, b)$ **of size** $n$ is the matrix $\mathbf{T}_{i,j}(b) \in \mathcal{M}_n(\mathbb{R})$ such that for all $k, l \in \{1, \dots, n\}$ :

$$
[\mathbf{T}_{i,j}(b)]_{k,l} := \begin{cases} b & \text{if } k = i \text{ and } l = j \\ \delta_{k,l} & \text{otherwise.} \end{cases}
$$

We denote by $\mathcal{E}_n(\mathbb{R})$ the set of elementary matrices of size $n$, i.e. :

$$
\begin{align*}
\mathcal{E}_n(\mathbb{R}) &= \{ \mathbf{P}_{i,j}, i,j \in \{1, \dots, n\}, i \neq j \} \\
&\quad \cup \{ \mathbf{D}_i(a), i \in \{1, \dots, n\}, a \in \mathbb{R}^* \} \\
&\quad \cup \{ \mathbf{T}_i(b), i,j \in \{1, \dots, n\}, i \neq j \wedge b \in \mathbb{R} \}.
\end{align*}
$$
For $n := 3$, for example :

$$ \mathbf{P}_{1,2} = \mathbf{P}_{2,1} = \begin{pmatrix} 0 & 1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{D}_1(a) = \begin{pmatrix} a & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{T}_{1,2}(b) = \begin{pmatrix} 1 & b & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{T}_{2,3}(b) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & b \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{P}_{1,3} = \mathbf{P}_{3,1} = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} $$

$$ \mathbf{D}_2(a) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & a & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{T}_{1,3}(b) = \begin{pmatrix} 1 & 0 & b \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{T}_{3,1}(b) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ b & 0 & 1 \end{pmatrix} $$

$$ \mathbf{P}_{2,3} = \mathbf{P}_{3,2} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix} $$

$$ \mathbf{D}_3(a) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & a \end{pmatrix} $$

$$ \mathbf{T}_{2,1}(b) = \begin{pmatrix} 1 & 0 & 0 \\ b & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ \mathbf{T}_{3,2}(b) = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & b & 1 \end{pmatrix} $$

## A.5 Properties

We have the following simple relations.

### Lemma A.5.1

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$, $a \in \mathbb{R}^*$, $b \in \mathbb{R}$, and $\sigma \in \mathfrak{S}_n$.

(i) (a) We have $\mathbf{P}_{i,j} \in \mathrm{GL}_n(\mathbb{R})$ and :

$$ \mathbf{P}_{i,j}^{-1} = \mathbf{P}_{i,j}^{T} = \mathbf{P}_{i,j}. $$

(b) Any permutation matrix is the product of at most $n-1$ elementary permutation matrices.

(c) We have :

$$ \det(\mathbf{P}_{\sigma}) = \varepsilon(\sigma). $$

(ii) (i) We have $\mathbf{D}_i(a) \in \mathrm{GL}_n(\mathbb{R})$ and :

$$ \mathbf{D}_i(a)^{-1} = \mathbf{D}_i(1/a) \ , \ \mathbf{D}_i(a)^T = \mathbf{D}_i(a). $$

(ii) We have :

$$ \det(\mathbf{D}_i(a)) = a. $$

(iii) (i) We have $\mathbf{T}_{i,j}(b) \in \mathrm{GL}_n(\mathbb{R})$ and :

$$ \mathbf{T}_{i,j}(b)^{-1} = \mathbf{T}_{i,j}(-b) \ , \ \mathbf{T}_{i,j}(b)^T = \mathbf{T}_{j,i}(b). $$

(ii) We have :

$$ \det(\mathbf{T}_{i,j}(b)) = 1. $$

### Proof.

(i) (a) By lemma A.3.3, we have $\mathbf{P}_{i,j} \in \mathrm{GL}_n(\mathbb{R})$ and :

$$ \begin{align*} \mathbf{P}_{i,j}^{-1} &= \mathbf{P}_{\tau_{i,j}}^{-1} \\ &= \mathbf{P}_{\tau_{i,j}}^{-1} \\ &= \mathbf{P}_{\tau_{i,j}} \end{align*} $$
$$ = \mathbf{P}_{i,j} $$

and :

$$
\begin{align*}
\mathbf{P}_{i,j}^T &= \mathbf{P}_{\tau_{i,j}}^T \\
&= \mathbf{P}_{\tau_{i,j}^{-1}} \\
&= \mathbf{P}_{\tau_{i,j}} \\
&= \mathbf{P}_{i,j}
\end{align*}
$$

(b) Let $\sigma \in \mathfrak{S}_n$ be a permutation. Then by proposition A.1.2, $\sigma$ is the product of transpositions $\tau_1, \dots, \tau_k$ :

$$ \sigma = \tau_1 \circ \dots \circ \tau_k. $$

Thus, by point (i) :

$$
\begin{align*}
\mathbf{P}_{\sigma} &= \mathbf{P}_{\tau_1 \circ \dots \circ \tau_k} \\
&= \mathbf{P}_{\tau_1} \cdots \mathbf{P}_{\tau_k}
\end{align*}
$$

Thus the result.

(c) Let $\tau_{i,j}$ be a transposition with $i, j \in \{1, \dots, n\}$ with $i < j$. We show that :

$$ \det(\mathbf{P}_{i,j}) = -1 = \varepsilon(\tau_{i,j}). $$

We have by the antisymmetry of the determinant :

$$
\begin{align*}
\det(\mathbf{P}_{i,j}) &= \det(e_1, \dots, e_{i-1}, e_j, e_{i+1}, \dots, e_{j-1}, e_i, e_{j+1}, \dots, e_n) \\
&= -\det(e_1, \dots, e_n) \\
&= -1 \\
&= \varepsilon(\tau_{i,j})
\end{align*}
$$

We reuse the notations from point (iii). Let a permutation decomposed into a product of transpositions :

$$ \sigma = \tau_1 \circ \dots \circ \tau_k. $$

We then have :

$$
\begin{align*}
\det(\mathbf{P}_{\sigma}) &= \det(\mathbf{P}_{\tau_1} \cdots \mathbf{P}_{\tau_k}) \\
&= \prod_{i=1}^{k} \det(\mathbf{P}_{\tau_i}) \\
&= (-1)^k \\
&= \varepsilon(\sigma)
\end{align*}
$$

(ii) Let $k, l \in \{1, \dots, n\}$. As :

$$ [\mathbf{D}_i(a)]_{k,l} = \begin{cases} a & \text{if } k=l=i \\ \delta_{k,l} & \text{otherwise.} \end{cases} = (a-1)\delta_{k,i}\delta_{l,i} + \delta_{k,l} $$

we have :

$$ [\mathbf{D}_i(a)\mathbf{D}_i(1/a)]_{kl} = \sum_{m=1}^{n} [\mathbf{D}_i(a)]_{km} [\mathbf{D}_i(1/a)]_{ml} $$
$$
\begin{align*}
&= \sum_{m=1}^{n} \left( (a-1)\delta_{k,i}\delta_{m,i} + \delta_{k,m} \right) \left( (1/a-1)\delta_{m,i}\delta_{l,i} + \delta_{m,l} \right) \\
&= (a-1)(1/a-1) \sum_{m=1}^{n} \delta_{k,i}\delta_{m,i}\delta_{m,i}\delta_{l,i} + (a-1)\sum_{m=1}^{n} \delta_{k,i}\delta_{m,i}\delta_{m,l} \\
&\quad + (1/a-1) \sum_{m=1}^{n} \delta_{k,m}\delta_{m,i}\delta_{l,i} + \sum_{m=1}^{n} \delta_{k,m}\delta_{m,l} \\
&= (a-1)(1/a-1)\delta_{k,i}\delta_{l,i} + (a-1)\delta_{k,i}\delta_{l,i} + (1/a-1)\delta_{k,i}\delta_{l,i} + \delta_{k,l} \\
&= \delta_{k,l} \\
&= [I_n]_{kl}
\end{align*}
$$

and we have :

$$
\begin{align*}
[\mathbf{D}_i(a)^T]_{kl} &= [\mathbf{D}_i(a)]_{lk} \\
&= (a-1)\delta_{l,i}\delta_{k,i} \\
&= (a-1)\delta_{k,i}\delta_{l,i} \\
&= [\mathbf{D}_i(a)]_{kl}
\end{align*}
$$

Thus we have $\mathbf{D}_i(a) \in \mathrm{GL}_n(\mathbb{R})$ and :

$$
\mathbf{D}_i(a)^{-1} = \mathbf{D}_i(1/a) , \quad \mathbf{D}_i(a)^T = \mathbf{D}_i(a).
$$

(iii) Let $k, l \in \{1, \dots, n\}$. Let us give an expression for the coefficients of these matrices without conditioning. We have for all $k, l \in \{1, \dots, n\}$ :

$$
[\mathbf{T}_{i,j}(b)]_{k,l} = \begin{cases} b & \text{if } k=i \text{ and } l=j \\ \delta_{k,l} & \text{otherwise.} \end{cases} = b\delta_{k,i}\delta_{l,j} + \delta_{k,l}
$$

we have :

$$
\begin{align*}
[\mathbf{T}_{i,j}(b)\mathbf{T}_{i,j}(-b)]_{kl} &= \sum_{m=1}^{n} [\mathbf{T}_{i,j}(b)]_{km} [\mathbf{T}_{i,j}(-b)]_{ml} \\
&= \sum_{m=1}^{n} (b\delta_{k,i}\delta_{m,j} + \delta_{k,m}) (-b\delta_{m,i}\delta_{l,j} + \delta_{m,l}) \\
&= -b^2 \sum_{m=1}^{n} \delta_{k,i}\delta_{m,j}\delta_{m,i}\delta_{l,j} + b \sum_{m=1}^{n} \delta_{k,i}\delta_{m,j}\delta_{m,l} - b \sum_{m=1}^{n} \delta_{k,m}\delta_{m,i}\delta_{l,j} + \sum_{m=1}^{n} \delta_{k,m}\delta_{m,l} \\
&= 0 + b\delta_{k,i}\delta_{j,l} - b\delta_{k,i}\delta_{l,j} + \delta_{k,l} \\
&= \delta_{k,l} \\
&= [I_n]_{kl}
\end{align*}
$$

and we have :

$$
\begin{align*}
[\mathbf{T}_{i,j}(b)^T]_{kl} &= [\mathbf{T}_{i,j}(b)]_{lk} \\
&= b\delta_{l,i}\delta_{k,j} + \delta_{l,k} \\
&= b\delta_{k,j}\delta_{l,i} + \delta_{k,l} \\
&= [\mathbf{T}_{j,i}(b)]_{kl}
\end{align*}
$$

Thus we have $\mathbf{T}_{i,j}(b) \in \mathrm{GL}_n(\mathbb{R})$ and :

$$
\mathbf{T}_{i,j}(b)^{-1} = \mathbf{T}_{i,j}(-b) , \quad \mathbf{T}_{i,j}(b)^T = \mathbf{T}_{j,i}(b).
$$

From this, we deduce two fundamental stabilities of the set $\mathscr{E}_n(\mathbb{R})$.

<figure><img src="image_12.png" /></figure>
Corollary A.5.2

The set $\mathcal{E}_n(\mathbb{R})$ of elementary matrices of size $n$ is stable under transposition and inversion.

A.6 Case of Triangular Matrices

This subsection shows that every invertible triangular matrix of size 3 can be written as a product of elementary operations. We start with lower triangular matrices.

Proposition A.6.1

(i) Let :

$$
L := \begin{pmatrix} 1 & 0 & 0 \\ b & 1 & 0 \\ d & e & 1 \end{pmatrix} \in \mathrm{GL}_3(\mathbb{R}).
$$

(a) We have :

$$
L = \mathbf{T}_{2,1}(b)\mathbf{T}_{3,1}(d)\mathbf{T}_{3,2}(e).
$$

(b) We have :

$$
\begin{align*}
L^{-1} &= \mathbf{T}_{3,2}(-e)\mathbf{T}_{3,1}(-d)\mathbf{T}_{2,1}(-b) \\
&= \begin{pmatrix}
1 & 0 & 0 \\
-b & 1 & 0 \\
be - d & -e & 1
\end{pmatrix}.
\end{align*}
$$

(ii) Let :

$$
L := \begin{pmatrix} a & 0 & 0 \\ b & c & 0 \\ d & e & f \end{pmatrix} \in \mathrm{GL}_3(\mathbb{R}).
$$

(so $a, c$, and $f$ are non-zero).

(a) We have :

$$
L = \mathbf{D}_3(f)\mathbf{D}_2(c)\mathbf{D}_1(a)\mathbf{T}_{2,1}(b/c)\mathbf{T}_{3,1}(d/f)\mathbf{T}_{3,2}(e/f).
$$

(b) We have :

$$
\begin{align*}
L^{-1} &= \mathbf{T}_{3,2}(-e/f)\mathbf{T}_{3,1}(-d/f)\mathbf{T}_{2,1}(-b/c)\mathbf{D}_1(1/a)\mathbf{D}_2(1/c)\mathbf{D}_3(1/f) \\
&= \begin{pmatrix}
1/a & 0 & 0 \\
-b/(ac) & 1/c & 0 \\
/be - cd/(acf) & -e/(cf) & 1/f
\end{pmatrix}.
\end{align*}
$$

Proof.

(i) (a) We have :

$$
\mathbf{T}_{2,1}(b)\mathbf{T}_{3,1}(d)\mathbf{T}_{3,2}(e) = \begin{pmatrix} 1 & 0 & 0 \\ b & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ d & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & e & 1 \end{pmatrix}
$$
$$
\begin{align*}
&= \begin{pmatrix} 1 & 0 & 0 \\ b & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ d & e & 1 \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ b & 1 & 0 \\ d & e & 1 \end{pmatrix} \\
&= L
\end{align*}
$$

(b) By (a) :

$$
\begin{align*}
L^{-1} &= \mathbf{T}_{3,2}(-e)\mathbf{T}_{3,1}(-d)\mathbf{T}_{2,1}(-b) \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -e & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -d & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -b & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -e & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -b & 1 & 0 \\ -d & 0 & 1 \end{pmatrix} \\
&= \begin{pmatrix} 1 & 0 & 0 \\ -b & 1 & 0 \\ be - d & -e & 1 \end{pmatrix}
\end{align*}
$$

(ii) Let :

$$
L := \begin{pmatrix} a & 0 & 0 \\ b & c & 0 \\ d & e & f \end{pmatrix} \in \mathrm{GL}_3(\mathbb{R}).
$$

(a) By (1.a) :

$$
\begin{align*}
& \mathbf{D}_3(f)\mathbf{D}_2(c)\mathbf{D}_1(a)\mathbf{T}_{2,1}(b/c)\mathbf{T}_{3,1}(d/f)\mathbf{T}_{3,2}(e/f) \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & f \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & c & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} a & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \bullet \begin{pmatrix} 1 & 0 & 0 \\ b/c & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ d/f & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & e/f & 1 \end{pmatrix} \\
&= \begin{pmatrix} a & 0 & 0 \\ 0 & c & 0 \\ 0 & 0 & f \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ b/c & 1 & 0 \\ d/f & e/f & 1 \end{pmatrix} \\
&= \begin{pmatrix} a & 0 & 0 \\ b & c & 0 \\ d & e & f \end{pmatrix} \\
&= L
\end{align*}
$$

(b) We have :

$$
\begin{align*}
L^{-1} &= \mathbf{T}_{3,2}(-e/f)\mathbf{T}_{3,1}(-d/f)\mathbf{T}_{2,1}(-b/c)\mathbf{D}_1(1/a)\mathbf{D}_2(1/c)\mathbf{D}_3(1/f) \\
&= \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & -e/f & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ -d/f & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ -b/c & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \bullet \begin{pmatrix} 1/a & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1/c & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1/f \end{pmatrix}
\end{align*}
$$
$$
\begin{align*}
&= \begin{pmatrix} 1 & 0 & 0 \\ -b/c & 1 & 0 \\ (be - dc)/(cf) & -e/f & 1 \end{pmatrix} \cdot \begin{pmatrix} 1/a & 0 & 0 \\ 0 & 1/c & 0 \\ 0 & 0 & 1/f \end{pmatrix} \\
&= \begin{pmatrix} 1/a & 0 & 0 \\ -b/(ac) & 1/c & 0 \\ (be - cd)/(acf) & -e/(cf) & 1/f \end{pmatrix}.
\end{align*}
$$

<figure><img src="image_13.png" /></figure>

By taking the transpose, we obtain the result for upper triangular matrices (see also Lemma A.5.1).

Proposition A.6.2

(i) Let :

$$
U := \begin{pmatrix} 1 & b & d \\ 0 & 1 & e \\ 0 & 0 & 1 \end{pmatrix} \in \mathrm{GL}_3(\mathbb{R}).
$$

(a) We have :

$$
U = \mathbf{T}_{2,3}(e)\mathbf{T}_{1,3}(d)\mathbf{T}_{1,2}(b).
$$

(b) We have :

$$
\begin{align*}
U^{-1} &= \mathbf{T}_{1,2}(-b)\mathbf{T}_{1,3}(-d)\mathbf{T}_{2,3}(-e) \\
&= \begin{pmatrix} 1 & -b & be-d \\ 0 & 1 & -e \\ 0 & 0 & 1 \end{pmatrix}.
\end{align*}
$$

(ii) Let :

$$
U := \begin{pmatrix} a & b & d \\ 0 & c & e \\ 0 & 0 & f \end{pmatrix} \in \mathrm{GL}_3(\mathbb{R}).
$$

(a) The reals a, c, and f are non-zero.

(b) We have :

$$
U = \mathbf{T}_{2,3}(e/f)\mathbf{T}_{1,3}(d/f)\mathbf{T}_{1,2}(b/c)\mathbf{D}_{1}(a)\mathbf{D}_{2}(c)\mathbf{D}_{3}(f).
$$

(c) We have :

$$
\begin{align*}
U^{-1} &= \mathbf{D}_3(1/f)\mathbf{D}_2(1/c)\mathbf{D}_1(1/a)\mathbf{T}_{1,2}(-b/c)\mathbf{T}_{1,3}(-d/f)\mathbf{T}_{2,3}(-e/f) \\
&= \begin{pmatrix} 1/a & -b/(ac) & (be - cd)/(acf) \\ 0 & 1/c & -e/(cf) \\ 0 & 0 & 1/f \end{pmatrix}.
\end{align*}
$$

The result generalizes to any dimension. Since matrix multiplication is not commutative, we use the notations :

$$
\prod_{i}^{1 \to n} a_i = a_1 \cdots a_n
$$

$$
\prod_{i}^{n \to 1} a_i = a_n \cdots a_1
$$

For lower and upper triangular matrices, the following result holds.
**Proposition A.6.3**

(i) Let :

$$
L := \begin{pmatrix} t_{11} & & (0) \\ \vdots & \ddots & \\ t_{n1} & & t_{n,n} \end{pmatrix} \in \mathrm{GL}_n(\mathbb{R}).
$$

(a) We have :

$$
L = \left[ \prod_{i=1}^{n} \mathbf{D}_i (t_{i,i}) \right] \cdot \left[ \prod_{i}^{2 \to n} \left[ \prod_{k=1}^{i-1} \mathbf{T}_{i,k} \left( \frac{t_{i,k}}{t_{i,i}} \right) \right] \right].
$$

(b) We have :

$$
L^{-1} = \prod_{i}^{n \to 2} \left[ \prod_{k=1}^{i-1} \mathbf{T}_{i,k} \left( -\frac{t_{i,k}}{t_{i,i}} \right) \right] \cdot \left[ \prod_{i=1}^{n} \mathbf{D}_i \left( \frac{1}{t_{i,i}} \right) \right].
$$

(ii) Let :

$$
U := \begin{pmatrix} t_{11} & \cdots & t_{1n} \\ \vdots & \ddots & \vdots \\ (0) & & t_{n,n} \end{pmatrix} \in \mathrm{GL}_n(\mathbb{R}).
$$

(a) We have :

$$
U = \left[ \prod_{i}^{n \to 2} \left[ \prod_{k=1}^{i-1} \mathbf{T}_{k,i} \left( \frac{t_{k,i}}{t_{i,i}} \right) \right] \cdot \left[ \prod_{i=1}^{n} \mathbf{D}_i (t_{i,i}) \right] \right].
$$

(b) We have :

$$
U^{-1} = \left[ \prod_{i=1}^{n} \mathbf{D}_i \left( \frac{1}{t_{i,i}} \right) \right] \cdot \prod_{i}^{2 \to n} \left[ \prod_{k=1}^{i-1} \mathbf{T}_{k,i} \left( -\frac{t_{i,k}}{t_{i,i}} \right) \right].
$$
# B Elementary Row Operations

We define the three elementary row operations. Each operation is associated with an invertible matrix. We keep the notations from the previous section. We start with the definition of row equivalence.

## Definition B.0.1

Let $M, N \in \mathcal{M}_{n,p}(\mathbb{R})$. We say that $M$ and $N$ are **row equivalent** if there exists $P \in \text{GL}_n(\mathbb{R})$ such that :

$$M = PN$$

We then denote $M \sim_L N$.

The relation $\sim_L$ is an equivalence relation on the matrices of $\mathcal{M}_{n,p}(\mathbb{R})$.

## B.1 The Three Elementary Operations

### Permutation Matrices

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$. The **permutation matrix** for swapping rows $i$ and $j$ of size $n$ is the matrix $\mathbf{P}_{i,j}$. It is denoted by :

$$L_i \leftrightarrow L_j.$$

We then denote :

$$\begin{align*} \text{Mat}(L_i \leftrightarrow L_j) &:= \mathbf{P}_{i,j} \\ \text{Op}_L(\mathbf{P}_{i,j}) &:= [L_i \leftrightarrow L_j] \end{align*}$$

### Dilation Matrices

Let $i \in \{1, \dots, n\}$ and $a \in \mathbb{R}^*$. The **dilation matrix** for scaling row $i$ by $a$ of size $n$ is the matrix $\mathbf{D}_i(a)$. It is denoted by :

$$L_i \leftarrow aL_i.$$

We then denote :

$$\begin{align*} \text{Mat}(L_i \leftarrow aL_i) &:= \mathbf{D}_i(a) \\ \text{Op}_L(\mathbf{D}_i(a)) &:= [L_i \leftarrow aL_i] \end{align*}$$

### Transvection Matrices

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$ and $b \in \mathbb{R}$. The **transvection matrix** $(b, j)$ for row $i$ of size $n$ is the matrix $\mathbf{T}_{i,j}(b, j)$. It is denoted by :

$$L_i \leftarrow L_i + bL_j.$$

We then denote :

$$\begin{align*} \text{Mat}(L_i \leftarrow L_i + bL_j) &:= \mathbf{T}_{i,j}(b, j) \\ \text{Op}_L(\mathbf{T}_{i,j}(b, j)) &:= [L_i \leftarrow L_i + bL_j] \end{align*}$$
## B.2 A Simple Example

Performing an elementary row operation on a matrix is equivalent to multiplying the matrix on the left by an elementary matrix.

Let's see this with a simple example. Consider the matrix :

$$M := \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix}.$$

(1) Dilation :

$[L_2 \leftarrow 5L_2]$

$$\mathbf{D}_2(5)M = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 20 & 25 & 30 \\ 7 & 8 & 9 \end{pmatrix}.$$

(2) Permutation :

(a)

$[L_2 \leftrightarrow L_3]$

$$\mathbf{P}_{2,3}M = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix} \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 7 & 8 & 9 \\ 4 & 5 & 6 \end{pmatrix}$$

(b)

$[L_1 \leftrightarrow L_3]$

$$\mathbf{P}_{1,3}M = \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} = \begin{pmatrix} 7 & 8 & 9 \\ 4 & 5 & 6 \\ 1 & 2 & 3 \end{pmatrix}$$

(3) Transvection :

$[L_3 \leftarrow L_3 + 3L_1]$

$$\mathbf{T}_{3,1}(3)M = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 3 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 10 & 14 & 18 \end{pmatrix}$$

(4) Transvection and Dilation :

$[L_3 \leftarrow 2L_3 + 3L_1]$

$$\mathbf{D}_3(2)\mathbf{T}_{3,1}(3/2)M = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 3 & 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 26 & 31 & 36 \end{pmatrix}$$

This corresponds to performing the two elementary operations successively :

$$[L_3 \leftarrow 2L_3 + 3L_1] \equiv [L_3 \leftarrow L_3 + 3/2 \cdot L_1 \quad \text{then} \quad L_3 \leftarrow 2L_3]$$

i.e., in matrix form <fnref n="8" />:

$$\begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 3 & 0 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 3/2 & 0 & 1 \end{pmatrix}.$$

<footnote n="8">8. The order in the matrix product is reversed because we multiply on the left.</footnote>
# C Elementary Operations on Columns

We define the three elementary operations on columns. Each is associated with an invertible matrix. They are simply the transposes of the elementary matrices on rows. We keep the notations from the previous sections. We begin with the definition of column equivalence.

## Definition C.0.1

Let $M, N \in \mathcal{M}_{p,n}(\mathbb{R})$. We say that $M$ and $N$ are column-equivalent if there exists $Q \in \mathrm{GL}_n(\mathbb{R})$ such that :

$$M = NQ^{-1}$$

We then write $M \sim_C N$.

The relation $\sim_C$ is an equivalence relation on the matrices of $\mathcal{M}_{p,n}(\mathbb{R})$.

## C.1 The Three Elementary Operations

### Permutation Matrices

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$. The **permutation matrix** of columns $i$ and $j$ of size $n$ is the matrix :

$$\mathbf{P}_{i,j}^{T} = \mathbf{P}_{i,j}.$$

It is symbolized by :

$$C_i \leftrightarrow C_j.$$

We then write :

$$\mathrm{Mat}(C_i \leftrightarrow C_j) := \mathbf{P}_{i,j}$$

$$\mathrm{Op}_C(\mathbf{P}_{i,j}) := [C_i \leftrightarrow C_j]$$

### Dilation Matrices

Let $i \in \{1, \dots, n\}$ and $a \in \mathbb{R}^*$. The **dilation matrix** of column $i$ by $a$ of size $n$ is :

$$\mathbf{D}_i(a)^T = \mathbf{D}_i(a).$$

It is symbolized by :

$$C_i \leftrightarrow aC_i.$$

We then write :

$$\mathrm{Mat}(C_i \leftrightarrow aC_i) := \mathbf{D}_i(a)$$

$$\mathrm{Op}_C(\mathbf{D}_i(a)) := [C_i \leftrightarrow aC_i]$$

### Transvection Matrices

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$ and $b \in \mathbb{R}$. The **transvection matrix** $(b, j)$ of column $i$ of size $n$ is :

$$\mathbf{T}_{i,j}(b, j)^T = \mathbf{T}_{j,i}(b, j).$$

It is symbolized by :

$$C_i \leftrightarrow C_i + bC_j.$$

We then write :

$$\mathrm{Mat}(C_i \leftrightarrow C_i + bC_j) := \mathbf{T}_{j,i}(b, j)$$

$$\mathrm{Op}_C(\mathbf{T}_{j,i}(b, j)) := [C_i \leftrightarrow C_i + bC_j]$$
## C.2 A Simple Example

Performing an elementary operation on the columns of a matrix is equivalent to multiplying the matrix on the right by an elementary matrix.

Let's look at this with a simple example. Let the matrix be:

$$M := \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix}.$$

(1) Dilation :

$$[C_2 \leftarrow 5C_2]$$

$$M D_2(5) = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 5 & 0 \\ 0 & 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 10 & 3 \\ 4 & 25 & 6 \\ 7 & 40 & 9 \end{pmatrix}.$$

(2) Permutation :

(a)

$$[C_2 \leftrightarrow C_3]$$

$$M P_{2,3} = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 0 & 1 \\ 0 & 1 & 0 \end{pmatrix} = \begin{pmatrix} 1 & 3 & 2 \\ 4 & 6 & 5 \\ 7 & 9 & 8 \end{pmatrix}$$

(b)

$$[C_1 \leftrightarrow C_3]$$

$$M P_{1,3} = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} \begin{pmatrix} 0 & 0 & 1 \\ 0 & 1 & 0 \\ 1 & 0 & 0 \end{pmatrix} = \begin{pmatrix} 3 & 2 & 1 \\ 6 & 5 & 4 \\ 9 & 8 & 7 \end{pmatrix}$$

(3) Transvection :

$$[C_3 \leftarrow C_3 + 3C_1]$$

$$M T_{3,1}(3) = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} \begin{pmatrix} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 6 \\ 4 & 5 & 18 \\ 7 & 8 & 30 \end{pmatrix}$$

(4) Transvection and dilation :

$$[C_3 \leftarrow 2C_3 + 3C_1]$$

$$M T_{3,1}(3/2) D_3(2) = \begin{pmatrix} 1 & 2 & 3 \\ 4 & 5 & 6 \\ 7 & 8 & 9 \end{pmatrix} \begin{pmatrix} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 2 & 9 \\ 4 & 5 & 24 \\ 7 & 8 & 39 \end{pmatrix}$$

This corresponds to the two elementary operations performed successively :

$$[C_3 \leftarrow 2C_3 + 3C_1] \equiv [C_3 \leftarrow C_3 + 3/2 \cdot C_1 \quad \text{then} \quad C_3 \leftarrow 2C_3]$$

i.e., we have in matrix terms :

$$\begin{pmatrix} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{pmatrix} = \begin{pmatrix} 1 & 0 & 3/2 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix} \begin{pmatrix} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 2 \end{pmatrix}.$$
# D Givens Rotations

In the QR decomposition, there is another method known for calculating the matrix Q. It uses Givens rotations. This method, which may seem complicated at first, is very useful in practice because it requires little computation to obtain the matrix Q.

In sections 13 and 14, we will find two other methods for calculating a matrix Q. This section also provides an opportunity, using the matrix A from the introduction, to show that the matrices Q and R are not unique : we will find matrices different from those in section 13 using the Gram-Schmidt method.

## D.1 Introduction

We begin with the definition of Givens rotations.

### Definition D.1.1

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$ and $\theta \in \mathbb{R}$. The **Givens rotation matrix** $G(i, j, \theta)$ **of size** $n$ is the orthogonal matrix of size $n$ given for all $k, l \in \{1, \dots, n\}$ by :

$$ [G(i, j, \theta)]_{kl} := \begin{cases} 1 & \text{if } k = l \neq i, j \\ \cos(\theta) & \text{if } k = l = i \text{ or } k = l = j \\ \sin(\theta) & \text{if } (k, l) = (j, i) \\ -\sin(\theta) & \text{if } (k, l) = (i, j) \end{cases} $$

It is also denoted as :

$$ G(i, j, c, s) := G(i, j, \theta) $$

with $c := \cos(\theta)$ and $s := \sin(\theta)$.

Thus :

$$ G(i, j, c, s) = G(i, j, \theta) = \begin{pmatrix} i & \cdots & i & \cdots & i \\ \vdots & \ddots & \vdots & \ddots & \vdots \\ 0 & \cdots & c & \cdots & 0 \\ \vdots & \ddots & \vdots & \ddots & \vdots \\ 0 & \cdots & s & \cdots & 0 \\ \vdots & \ddots & \vdots & \ddots & \vdots \\ 0 & \cdots & 0 & \cdots & 1 \end{pmatrix} \begin{pmatrix} i \\ \vdots \\ j \\ \vdots \\ j \\ \vdots \end{pmatrix} $$

### Example D.1.2: Case $n := 3$

Let $\theta \in \mathbb{R}$. Set $c := \cos(\theta)$ and $s := \sin(\theta)$. The 6 types of Givens rotations are given by :

$$ G(1, 2, c, s) = G(1, 2, \theta) = \begin{pmatrix} c & -s & 0 \\ s & c & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ G(2, 1, c, s) = G(2, 1, \theta) = \begin{pmatrix} -s & c & 0 \\ c & s & 0 \\ 0 & 0 & 1 \end{pmatrix} $$

$$ G(1, 3, c, s) = G(1, 3, \theta) = \begin{pmatrix} c & 0 & -s \\ s & 0 & c \\ 0 & 1 & 0 \end{pmatrix} $$

$$ G(3, 1, c, s) = G(3, 1, \theta) = \begin{pmatrix} -s & 0 & c \\ c & 0 & s \\ 0 & 1 & 0 \end{pmatrix} $$
$$
G(2, 3, c, s) = G(2, 3, \theta) = \begin{pmatrix} 0 & c & -s \\ 0 & s & c \\ 1 & 0 & 0 \end{pmatrix}
$$

$$
G(3, 2, c, s) = G(3, 2, \theta) = \begin{pmatrix} 0 & -s & c \\ 0 & c & s \\ 1 & 0 & 0 \end{pmatrix}
$$

We start with a useful lemma.

Lemma D.1.3

Let $i, j \in \{1, \dots, n\}$ such that $i \neq j$ and $\theta \in \mathbb{R}$.

(i) We have $G(i, j, \theta) \in O_n(\mathbb{R})$.

(ii) Let :

$$
X := \begin{pmatrix} x_1 \\ \vdots \\ x_n \end{pmatrix} \in \mathbb{R}^n
$$

with $r := \sqrt{x_i^2 + x_j^2} \neq 0$. Then :

$$
G\left(i, j, \frac{x_i}{r}, -\frac{x_j}{r}\right) X = \begin{pmatrix} x_1 \\ \vdots \\ r \\ \vdots \\ 0 \\ \vdots \\ x_n \end{pmatrix} \begin{matrix} i \\ j \end{matrix}
$$

**Proof.** In both proofs, we will use an expression of $G(i, j, \theta)$ with Kronecker symbols. For all $k, l \in \{1, \dots, n\}$, we have :

$$
[G(i, j, \theta)]_{kl} = s(\delta_{(k,l),(j,i)} - \delta_{(k,l),(i,j)}) + (c-1)(\delta_{(k,l),(i,i)} + \delta_{(k,l),(j,j)}) + \delta_{k,l}
$$

(i) Let $c := \cos(\theta)$ and $s := \sin(\theta)$. We will show that $G(i, j, \theta)^T G(i, j, \theta) = I_n$. For all $k, l \in \{1, \dots, n\}$, and since $s^2 + c^2 = 1$, we have :

$$
\begin{align*}
& [G(i, j, \theta)^T G(i, j, \theta)]_{kl} \\
&= \sum_{h=1}^{n} [G(i, j, \theta)^T]_{kh} [G(i, j, \theta)]_{hl} \\
&= \sum_{h=1}^{n} [G(i, j, \theta)]_{hk} [G(i, j, \theta)]_{hl} \\
&= \sum_{h=1}^{n} \left( s(\delta_{(h,k),(j,i)} - \delta_{(h,k),(i,j)}) + (c-1)(\delta_{(h,k),(i,i)} + \delta_{(h,k),(j,j)}) + \delta_{h,k} \right) \\
&\qquad \times \left( s(\delta_{(h,l),(j,i)} - \delta_{(h,l),(i,j)}) + (c-1)(\delta_{(h,l),(i,i)} + \delta_{(h,l),(j,j)}) + \delta_{h,l} \right) \\
&= s^2 \sum_{h=1}^{n} (\delta_{(h,k),(j,i)} - \delta_{(h,k),(i,j)})(\delta_{(h,l),(j,i)} - \delta_{(h,l),(i,j)}) \\
&\qquad + (c-1)^2 \sum_{h=1}^{n} (\delta_{(h,k),(i,i)} + \delta_{(h,k),(j,j)})(\delta_{(h,l),(i,i)} + \delta_{(h,l),(j,j)}) \\
&\qquad + s(c-1) \sum_{h=1}^{n} \left[ (\delta_{(h,k),(j,i)} - \delta_{(h,k),(i,j)})(\delta_{(h,l),(i,i)} + \delta_{(h,l),(j,j)}) + (\delta_{(h,k),(i,i)} + \delta_{(h,k),(j,j)})(\delta_{(h,l),(j,i)} - \delta_{(h,l),(i,j)}) \right]
\end{align*}
$$