# Linear Algebra/Gauss-Jordan Reduction

|current=Gauss-Jordan Reduction
|previous=Reduced Echelon Form
elimination coupled with back-substitution solves linear systems, but
it\'s not the only method possible. Here is an extension of Gauss\'
method that has some advantages.

Note that the pivot operations in the first stage proceed from column
one to column three while the pivot operations in the third stage
proceed from column three to column one.

This extension of Gauss\' method is **Gauss-Jordan reduction**. It goes
past echelon form to a more refined, more specialized, matrix form.

The disadvantage of using Gauss-Jordan reduction to solve a system is
that the additional row operations mean additional arithmetic. The
advantage is that the solution set can just be read off.

In any echelon form, plain or reduced, we can read off when a system has
an empty solution set because there is a contradictory equation, we can
read off when a system has a one-element solution set because there is
no contradiction and every variable is the leading variable in some row,
and we can read off when a system has an infinite solution set because
there is no contradiction and at least one variable is free.

In reduced echelon form we can read off not just what kind of solution
set the system has, but also its description. Whether or not the echelon
form is reduced, we have no trouble describing the solution set when it
is empty, of course. The two examples above show that when the system
has a single solution then the solution can be read off from the
right-hand column. In the case when the solution set is infinite, its
parametrization can also be read off of the reduced echelon form.
Consider, for example, this system that is shown brought to echelon form
and then to reduced echelon form.

$$\left(\begin{array}{cccc|c}
2  &6  &1  &2  &5  \\
0  &3  &1  &4  &1  \\
0  &3  &1  &2  &5
\end{array}\right)
\xrightarrow[]{-\rho_2+\rho_3}
\left(\begin{array}{cccc|c}
2  &6  &1  &2  &5  \\
0  &3  &1  &4  &1  \\
0  &0  &0  &-2 &4
\end{array}\right)$$

$$\xrightarrow[\begin{array}{c}\\[-19pt]\scriptstyle (1/3)\rho_2 \\[-5pt]\scriptstyle -(1/2)\rho_3\end{array}]{(1/2)\rho_1}
\;\xrightarrow[-\rho_3+\rho_1]{(4/3)\rho_3+\rho_2}
\;\xrightarrow[]{-3\rho_2+\rho_1}
\left(\begin{array}{cccc|c}
1  &0  &-1/2  &0  &-9/2  \\
0  &1  &1/3   &0  &3  \\
0  &0  &0     &1  &-2
\end{array}\right)$$

Starting with the middle matrix, the echelon form version, back
substitution produces $-2x_4=4$ so that $x_4=-2$, then another back
substitution gives $3x_2+x_3+4(-2)=1$ implying that $x_2=3-(1/3)x_3$,
and then the final back substitution gives
$2x_1+6(3-(1/3)x_3)+x_3+2(-2)=5$ implying that $x_1=-(9/2)+(1/2)x_3$.
Thus the solution set is this.

$$S=\{\begin{pmatrix} x_1 \\ x_2 \\ x_3 \\ x_4 \end{pmatrix}
=\begin{pmatrix} -9/2 \\ 3 \\ 0 \\ -2 \end{pmatrix}
+\begin{pmatrix} 1/2 \\ -1/3 \\ 1 \\ 0 \end{pmatrix}x_3
\,\big|\, x_3\in\mathbb{R}\}$$

Now, considering the final matrix, the reduced echelon form version,
note that adjusting the parametrization by moving the $x_3$ terms to the
other side does indeed give the description of this infinite solution
set.

Part of the reason that this works is straightforward. While a set can
have many parametrizations that describe it, e.g., both of these also
describe the above set $S$ (take $t$ to be $x_3/6$ and $s$ to be
$x_3-1$)

$$\{\begin{pmatrix} -9/2 \\ 3 \\ 0 \\ -2 \end{pmatrix}
+\begin{pmatrix} 3 \\ -2 \\ 6 \\ 0 \end{pmatrix}t
\,\big|\, t\in\mathbb{R}\}
\qquad
\{\begin{pmatrix} -4 \\ 8/3 \\ 1 \\ -2 \end{pmatrix}
+\begin{pmatrix} 1/2 \\ -1/3 \\ 1 \\ 0 \end{pmatrix}s
\,\big|\, s\in\mathbb{R}\}$$

nonetheless we have in this book stuck to a convention of parametrizing
using the unmodified free variables (that is, $x_3=x_3$ instead of
$x_3=6t$). We can easily see that a reduced echelon form version of a
system is equivalent to a parametrization in terms of unmodified free
variables. For instance,

$$\begin{array}{rl}
x_1 &=4-2x_3 \\
x_2 &=3-x_3
\end{array}
\quad\Longleftrightarrow\quad
\left(\begin{array}{ccc|c}
1  &0  &2  &4  \\
0  &1  &1  &3  \\
0  &0  &0  &0
\end{array}\right)$$

(to move from left to right we also need to know how many equations are
in the system). So, the convention of parametrizing with the free
variables by solving each equation for its leading variable and then
eliminating that leading variable from every other equation is exactly
equivalent to the reduced echelon form conditions that each leading
entry must be a one and must be the only nonzero entry in its column.

Not as straightforward is the other part of the reason that the reduced
echelon form version allows us to read off the parametrization that we
would have gotten had we stopped at echelon form and then done back
substitution. The prior paragraph shows that reduced echelon form
corresponds to some parametrization, but why the same parametrization? A
solution set can be parametrized in many ways, and Gauss\' method or the
Gauss-Jordan method can be done in many ways, so a first guess might be
that we could derive many different reduced echelon form versions of the
same starting system and many different parametrizations. But we never
do. Experience shows that starting with the same system and proceeding
with row operations in many different ways always yields the same
reduced echelon form and the same parametrization (using the unmodified
free variables).

In the rest of this section we will show that the reduced echelon form
version of a matrix is unique. It follows that the parametrization of a
linear system in terms of its unmodified free variables is unique
because two different ones would give two different reduced echelon
forms.

We shall use this result, and the ones that lead up to it, in the rest
of the book but perhaps a restatement in a way that makes it seem more
immediately useful may be encouraging. Imagine that we solve a linear
system, parametrize, and check in the back of the book for the answer.
But the parametrization there appears different. Have we made a mistake,
or could these be different-looking descriptions of the same set, as
with the three descriptions above of $S$? The prior paragraph notes that
we will show here that different-looking parametrizations (using the
unmodified free variables) describe genuinely different sets.

Here is an informal argument that the reduced echelon form version of a
matrix is unique. Consider again the example that started this section
of a matrix that reduces to three different echelon form matrices. The
first matrix of the three is the natural echelon form version. The
second matrix is the same as the first except that a row has been
halved. The third matrix, too, is just a cosmetic variant of the first.
The definition of reduced echelon form outlaws this kind of fooling
around. In reduced echelon form, halving a row is not possible because
that would change the row\'s leading entry away from one, and neither is
combining rows possible, because then a leading entry would no longer be
alone in its column.

This informal justification is not a proof; we have argued that no two
different reduced echelon form matrices are related by a single row
operation step, but we have not ruled out the possibility that multiple
steps might do. Before we go to that proof, we finish this subsection by
rephrasing our work in a terminology that will be enlightening.

Many different matrices yield the same reduced echelon form matrix. The
three echelon form matrices from the start of this section, and the
matrix they were derived from, all give this reduced echelon form
matrix.

$$\begin{pmatrix}
1  &0  \\
0  &1
\end{pmatrix}$$

We think of these matrices as related to each other. The next result
speaks to this relationship.

This lemma suggests that \"reduces to\" is misleading--- where
$A\longrightarrow B$, we shouldn\'t think of $B$ as \"after\" $A$ or
\"simpler than\" $A$. Instead we should think of them as interreducible
or interrelated. Below is a picture of the idea. The matrices from the
start of this section and their reduced echelon form version are shown
in a cluster. They are all interreducible; these relationships are shown
also.

```{=html}
<center>
```
![](Linalg_interreducible_matrices.png "Linalg_interreducible_matrices.png"){width=""
height="200"}

```{=html}
</center>
```
We say that matrices that reduce to each other are \"equivalent with
respect to the relationship of row reducibility\". The next result
verifies this statement using the definition of an equivalence.[^1]

The diagram below shows the collection of all matrices as a box. Inside
that box, each matrix lies in some class. Matrices are in the same class
if and only if they are interreducible. The classes are disjoint--- no
matrix is in two distinct classes.
The collection of
matrices has been partitioned into **row equivalence classes**.[^2]

```{=html}
<center>
```
![](Linalg_row_equiv_classes.png "Linalg_row_equiv_classes.png"){width=""
height="200"}

```{=html}
</center>
```
One of the classes in this partition is the cluster of matrices shown
above, expanded to include all of the nonsingular $2 \! \times \! 2$
matrices.

The next subsection proves that the reduced echelon form of a matrix is
unique; that every matrix reduces to one and only one reduced echelon
form matrix. Rephrased in terms of the row-equivalence relationship, we
shall prove that every matrix is row equivalent to one and only one
reduced echelon form matrix. In terms of the partition what we shall
prove is: every equivalence class contains one and only one reduced
echelon form matrix. So each reduced echelon form matrix serves as a
representative of its class.

After that proof we shall, as mentioned in the introduction to this
section, have a way to decide if one matrix can be derived from another
by row reduction. We just apply the Gauss-Jordan procedure to both and
see whether or not they come to the same reduced echelon form.

## Exercises

## Footnotes

```{=html}
<references/>
```
|current=Gauss-Jordan Reduction
|previous=Reduced Echelon Form
|next=Row Equivalence}}`{=mediawiki}

[^1]: More information on equivalence relations is in the appendix.

[^2]: More information on partitions and class representatives is in the
    appendix.


# Linear Algebra/Row Equivalence

|current=Row Equivalence
|previous=Gauss-Jordan Reduction
We will close this section and this chapter by proving that every matrix
is row equivalent to one and only one reduced echelon form matrix. The
ideas that appear here will reappear, and be further developed, in the
next chapter.

The underlying theme here is that one way to understand a mathematical
situation is by being able to classify the cases that can happen. We
have met this theme several times already. We have classified solution
sets of linear systems into the no-elements, one-element, and
infinitely-many elements cases. We have also classified linear systems
with the same number of equations as unknowns into the nonsingular and
singular cases. We adopted these classifications because they give us a
way to understand the situations that we were investigating. Here, where
we are investigating row equivalence, we know that the set of all
matrices breaks into the row equivalence classes. When we finish the
proof here, we will have a way to understand each of those classes---
its matrices can be thought of as derived by row operations from the
unique reduced echelon form matrix in that class.

To understand how row operations act to transform one matrix into
another, we consider the effect that they have on the parts of a matrix.
The crucial observation is that row operations combine the rows
linearly.

(We have already used the phrase \"linear combination\" in this book.
The meaning is unchanged, but the next result\'s statement makes a more
formal definition in order.)

In this subsection we will use the convention that, where a matrix is
named with an upper case roman letter, the matching lower-case greek
letter names the rows.

$$A=
\begin{pmatrix}
\cdots \alpha_1 \cdots \\
\cdots \alpha_2 \cdots \\
\vdots                 \\
\cdots \alpha_m \cdots
\end{pmatrix}
\qquad
B=
\begin{pmatrix}
\cdots \beta_1 \cdots \\
\cdots \beta_2 \cdots \\
\vdots                \\
\cdots \beta_m\cdots
\end{pmatrix}$$

The proof below uses induction on the number of row operations used to
reduce one matrix to the other. Before we proceed, here is an outline of
the argument (readers unfamiliar with induction may want to compare this
argument with the one used in the
\"$\text{General}=\text{Particular}+\text{Homogeneous}$\" proof).[^1]
First, for the base step of the argument, we will verify that the
proposition is true when reduction can be done in zero row operations.
Second, for the inductive step, we will argue that if being able to
reduce the first matrix to the second in some number $t\geq 0$ of
operations implies that each row of the second is a linear combination
of the rows of the first, then being able to reduce the first to the
second in $t+1$ operations implies the same thing. Together, this base
step and induction step prove this result because by the base step the
proposition is true in the zero operations case, and by the inductive
step the fact that it is true in the zero operations case implies that
it is true in the one operation case, and the inductive step applied
again gives that it is therefore true in the two operations case, etc.

The prior result gives us the insight that Gauss\' method works by
taking linear combinations of the rows. But to what end; why do we go to
echelon form as a particularly simple, or basic, version of a linear
system? The answer, of course, is that echelon form is suitable for back
substitution, because we have isolated the variables. For instance, in
this matrix

$$R=\begin{pmatrix}
2  &3  &7  &8  &0  &0  \\
0  &0  &1  &5  &1  &1  \\
0  &0  &0  &3  &3  &0  \\
0  &0  &0  &0  &2  &1
\end{pmatrix}$$

$x_1$ has been removed from $x_5$\'s equation. That is, Gauss\' method
has made $x_5$\'s row independent of $x_1$\'s row.

Independence of a collection of row vectors, or of any kind of vectors,
will be precisely defined and explored in the next chapter. But a first
take on it is that we can show that, say, the third row above is not
comprised of the other rows, that
$\rho_3\neq c_1\rho_1+c_2\rho_2+c_4\rho_4$. For, suppose that there are
scalars $c_1$, $c_2$, and $c_4$ such that this relationship holds.

$$\begin{array}{rl}
\begin{pmatrix} 0  &0  &0  &3  &3  &0 \end{pmatrix}
&=c_1\begin{pmatrix} 2 &3 &7 &8 &0 &0 \end{pmatrix}             \\
&\quad+c_2\begin{pmatrix} 0 &0 &1 &5 &1 &1 \end{pmatrix} \\
&\quad+c_4\begin{pmatrix} 0 &0 &0 &0 &2 &1 \end{pmatrix}
\end{array}$$

The first row\'s leading entry is in the first column and narrowing our
consideration of the above relationship to consideration only of the
entries from the first column $0=2c_1+0c_2+0c_4$ gives that $c_1=0$. The
second row\'s leading entry is in the third column and the equation of
entries in that column $0=7c_1+1c_2+0c_4$, along with the knowledge that
$c_1=0$, gives that $c_2=0$. Now, to finish, the third row\'s leading
entry is in the fourth column and the equation of entries in that column
$3=8c_1+5c_2+0c_4$, along with $c_1=0$ and $c_2=0$, gives an
impossibility.

The following result shows that this effect always holds. It shows that
what Gauss\' linear elimination method eliminates is linear
relationships among the rows.

We can now prove that each matrix is row equivalent to one and only one
reduced echelon form matrix. We will find it convenient to break the
first half of the argument off as a preliminary lemma. For one thing, it
holds for any echelon form whatever, not just reduced echelon form.

For the proof we rephrase the result in more technical terms.
Define the **form** of an
$m \! \times \! n$ matrix to be the sequence
$\langle \ell_1,\ell_2,\ldots\,,\ell_m \rangle$ where $\ell_i$ is the
column number of the leading entry in row $i$ and $\ell_i=\infty$ if
there is no leading entry in that row. The lemma says that if two
echelon form matrices are row equivalent then their forms are equal
sequences.

That lemma answers two of the questions that we have posed: (i) any two
echelon form versions of a matrix have the same free variables, and
consequently, and (ii) any two echelon form versions have the same
number of free variables. There is no linear system and no combination
of row operations such that, say, we could solve the system one way and
get $y$ and $z$ free but solve it another way and get $y$ and $w$ free,
or solve it one way and get two free variables while solving it another
way yields three.

We finish now by specializing to the case of reduced echelon form
matrices.

We end with a recap. In Gauss\' method we start with a matrix and then
derive a sequence of other matrices. We defined two matrices to be
related if one can be derived from the other. That relation is an
equivalence relation, called row equivalence, and so partitions the set
of all matrices into row equivalence classes.

```{=html}
<center>
```
![](Linalg_reduced_echelon_form_equiv_classes.png "Linalg_reduced_echelon_form_equiv_classes.png"){width=""
height="200"}

```{=html}
</center>
```
(There are infinitely many matrices in the pictured class, but we\'ve
only got room to show two.) We have proved there is one and only one
reduced echelon form matrix in each row equivalence class.
So the reduced echelon form is a
*canonical form*[^2] for row equivalence: the reduced echelon form
matrices are representatives of the classes.

```{=html}
<center>
```
![](Linalg_reduced_echelon_form_equiv_classes_2.png "Linalg_reduced_echelon_form_equiv_classes_2.png"){width=""
height="200"}

```{=html}
</center>
```
We can answer questions about the classes by translating them into
questions about the representatives.

## Exercises

## Footnotes

```{=html}
<references/>
```

## References

-
-

[^1]: More information on mathematical induction is in the appendix.

[^2]: More information on canonical representatives is in the appendix.


# Linear Algebra/Topic: Computer Algebra Systems

|current=Topic: Computer Algebra Systems
|previous=Row Equivalence
The linear systems in this chapter are small enough that their solution
by hand is easy. But large systems are easiest, and safest, to do on a
computer. There are special purpose
programs such as LINPACK for this job.
Another popular tool is a general purpose
computer algebra system, including both commercial packages such as
Maple, Mathematica, or MATLAB, or free packages such as SciLab, Sage, or
Octave.

For example, in the Topic on Networks, we need to solve this.

$$\begin{array}{*{7}{rc}r}
i_0  &-  &i_1  &-  &i_2  &   &    &  &    &   &    &  &    &=  &0  \\
&   &i_1  &   &     &-  &i_3 &  &    &-  &i_5 &  &    &=  &0  \\
&   &     &   &i_2  &   &    &- &i_4 &+  &i_5 &  &    &=  &0  \\
&   &     &   &     &   &i_3 &+ &i_4 &   &    &- &i_6 &=  &0  \\
&   &5i_1 &   &     &+  &10i_3  &  & &   &    &  &    &=  &10  \\
&   &     &   &2i_2 &   &    &+ &4i_4 &  &    &  &    &=  &10  \\
&   &5i_1 &-  &2i_2 &   &    &  &    &+  &50i_5 &&    &=  &0
\end{array}$$

It can be done by hand, but it would take a while and be error-prone.
Using a computer is better.

We illustrate by solving that system under Maple (for another system, a
user\'s manual would obviously detail the exact syntax needed). The
array of coefficients can be entered in this way

\
`> A:=array( [[1,-1,-1,0,0,0,0],`\
`[0,1,0,-1,0,-1,0],`\
`[0,0,1,0,-1,1,0],`\
`[0,0,0,1,1,0,-1],`\
`[0,5,0,10,0,0,0],`\
`[0,0,2,0,4,0,0],`\
`[0,5,-2,0,0,50,0]] );`\
\
(putting the rows on separate lines is not necessary, but is done for
clarity). The vector of constants is entered similarly.

\
`> u:=array( [0,0,0,0,10,10,0] );`\
\
Then the system is solved, like magic.

\
`> linsolve(A,u);`\
`7 2 5 2 5 7`\
`[ -, -, -, -, -, 0, - ]`\
`3 3 3 3 3 3`\
\
Mathematica can solve this with

``` mathematica
RowReduce[({{1, -1, -1, 0, 0, 0, 0, 0}, {0, 1, 0, -1, 0, -1, 0,
    0}, {0, 0, 1, 0, -1, 1, 0, 0}, {0, 0, 0, 1, 1, 0, -1, 0}, {0, 5,
    0, 10, 0, 0, 0, 10}, {0, 0, 2, 0, 4, 0, 0, 10}, {0, 5, -2, 0, 0,
    50, 0, 0}})]
```

This returns the following output:

``` mathematica
{{1, 0, 0, 0, 0, 0, 0, 7/3}, {0, 1, 0, 0, 0, 0, 0, 2/3}, {0, 0, 1, 0,
  0, 0, 0, 5/3}, {0, 0, 0, 1, 0, 0, 0, 2/3}, {0, 0, 0, 0, 1, 0, 0, 5/
  3}, {0, 0, 0, 0, 0, 1, 0, 0}, {0, 0, 0, 0, 0, 0, 1, 7/3}}
```

Systems with infinitely many solutions are solved in the same way--- the
computer simply returns a parametrization.

## Exercises

*Answers for this Topic use Maple as the computer algebra system. In
particular, all of these were tested on Maple V running under MS-DOS NT
version 4.0. (On all of them, the preliminary command to load the linear
algebra package along with Maple\'s responses to the Enter key, have
been omitted.) Other systems have similar commands.* `{{TextBox|1=
;Problem 1:
Use the computer to solve the two problems that opened this
chapter.

 This is the Statics problem.

:<math>\begin{array}{rl}
40h+15c  &= 100  \\
25c      &= 50+50h
\end{array}</math>

 This is the Chemistry problem.

:<math>\begin{array}{rl}
7h      &= 7j  \\
8h +1i  &= 5j+2k  \\
1i      &= 3j  \\
3i      &= 6j+1k
\end{array}</math>

}}`{=mediawiki} `{{TextBox|1=
;Problem 2:
Use the computer to solve these systems from the
first subsection,
or conclude "many solutions" or "no solutions".

 <math>
\begin{array}{*{2}{rc}r}
2x  &+  &2y  &=  &5  \\
x  &-  &4y  &=  &0
\end{array}
</math>

 <math>
\begin{array}{*{2}{rc}r}
-x  &+  &y   &=  &1  \\
x  &+  &y   &=  &2
\end{array}
</math>

  <math>
\begin{array}{*{3}{rc}r}
x  &-  &3y  &+  &z  &=  &1  \\
x  &+  &y   &+  &2z &=  &14
\end{array}
</math>

  <math>
\begin{array}{*{2}{rc}r}
-x  &-  &y   &=  &1  \\
-3x  &-  &3y  &=  &2
\end{array}
</math>

  <math>
\begin{array}{*{3}{rc}r}
&   &4y  &+  &z  &=  &20 \\
2x  &-  &2y  &+  &z  &=  &0  \\
x  &   &    &+  &z  &=  &5  \\
x  &+  &y   &-  &z  &=  &10
\end{array}
</math>

 <math> \begin{array}{*{4}{rc}r}
2x  &   &   &+  &z  &+  &w  &=  &5  \\
&   &y  &   &   &-  &w  &=  &-1 \\
3x  &   &   &-  &z  &-  &w  &=  &0  \\
4x  &+  &y  &+  &2z &+  &w  &=  &9
\end{array}
</math>

}}`{=mediawiki} `{{TextBox|1=
;Problem 3:
Use the computer to solve these systems from the second subsection.

 <math> \begin{array}{*{2}{rc}r}
3x  &+  &6y  &=  &18  \\
x  &+  &2y  &=  &6
\end{array}  </math>

 <math> \begin{array}{*{2}{rc}r}
x  &+  &y   &=  &1  \\
x  &-  &y   &=  &-1
\end{array}  </math>

 <math> \begin{array}{*{3}{rc}r}
x_1  &   &     &+  &x_3   &=  &4  \\
x_1  &-  &x_2  &+  &2x_3  &=  &5  \\
4x_1  &-  &x_2  &+  &5x_3  &=  &17
\end{array}  </math>

 <math> \begin{array}{*{3}{rc}r}
2a   &+  &b    &-  &c     &=  &2  \\
2a   &   &     &+  &c     &=  &3  \\
a   &-  &b    &   &      &=  &0
\end{array}  </math>

 <math> \begin{array}{*{4}{rc}r}
x  &+  &2y   &-   &z   &    &    &=  &3  \\
2x  &+  &y    &    &    &+   &w   &=  &4  \\
x  &-  &y    &+   &z   &+   &w   &=  &1
\end{array}  </math>

 <math> \begin{array}{*{4}{rc}r}
x  &   &     &+   &z   &+   &w   &=  &4  \\
2x  &+  &y    &    &    &-   &w   &=  &2  \\
3x  &+  &y    &+   &z   &    &    &=  &7
\end{array}  </math>

}}`{=mediawiki} `{{TextBox|1=
;Problem 4:
What does the computer give for the solution of the general
<math>2 \! \times \! 2</math>  system?

:<math>
\begin{array}{*{2}{rc}r}
ax  &+  &cy  &=  &p  \\
bx  &+  &dy  &=  &q
\end{array}
</math>
/Solutions/


# Linear Algebra/Topic: Input-Output Analysis

|current=Topic: Input-Output Analysis
|previous=Topic: Computer Algebra Systems
An economy is an immensely complicated network of interdependences.
Changes in one part can ripple out to affect other parts. Economists
have struggled to be able to describe, and to make predictions about,
such a complicated object. Mathematical models using systems of linear
equations have emerged as a key tool. One is Input-Output Analysis,
pioneered by W. Leontief, who won the 1973 Nobel Prize in Economics.

Consider an economy with many parts, two of which are the steel industry
and the auto industry. As they work to meet the demand for their product
from other parts of the economy, that is, from users external to the
steel and auto sectors, these two interact tightly. For instance, should
the external demand for autos go up, that would lead to an increase in
the auto industry\'s usage of steel. Or, should the external demand for
steel fall, then it would lead to a fall in steel\'s purchase of autos.
The type of Input-Output model we will consider takes in the external
demands and then predicts how the two interact to meet those demands.

We start with a listing of production and consumption statistics. (These
numbers, giving dollar values in millions, are excerpted from
, describing the 1958 U.S. economy.
Today\'s statistics would be quite different, both because of inflation
and because of technical changes in the industries.)

```{=html}
<center>
```
+-----------------+----------------+----------------+----------------+-------------+
|                 |   *used by*  \ |   *used by*  \ |   *used by*  \ |   *total*   |
|                 |   *steel*      |   *auto*       |   *others*     |             |
+-----------------+----------------+----------------+----------------+-------------+
|   *value of*  \ |   5 395        |   2 664        |                |   25 448    |
|   *steel*       |                |                |                |             |
+-----------------+----------------+----------------+----------------+-------------+
|   *value of*  \ |   48           |   9 030        |                |   30 346    |
|   *auto*        |                |                |                |             |
+-----------------+----------------+----------------+----------------+-------------+

```{=html}
</center>
```
For instance, the dollar value of steel used by the auto industry in
this year is $2,664$ million. Note that industries may consume some of
their own output.

We can fill in the blanks for the external demand. This year\'s value of
the steel used by others this year is $17,389$ and this year\'s value of
the auto used by others is $21,268$. With that, we have a complete
description of the external demands and of how auto and steel interact,
this year, to meet them.

Now, imagine that the external demand for steel has recently been going
up by $200$ per year and so we estimate that next year it will be
$17,589$. Imagine also that for similar reasons we estimate that next
year\'s external demand for autos will be down $25$ to $21,243$. We wish
to predict next year\'s total outputs.

That prediction isn\'t as simple as adding $200$ to this year\'s steel
total and subtracting $25$ from this year\'s auto total. For one thing,
a rise in steel will cause that industry to have an increased demand for
autos, which will mitigate, to some extent, the loss in external demand
for autos. On the other hand, the drop in external demand for autos will
cause the auto industry to use less steel, and so lessen somewhat the
upswing in steel\'s business. In short, these two industries form a
system, and we need to predict the totals at which the system as a whole
will settle.

For that prediction, let $s$ be next years total production of steel and
let $a$ be next year\'s total output of autos. We form these equations.

$$\begin{array}{rl}
\text{next year}\textrm{'}\text{s production of steel}
&=\text{next year}\textrm{'}\text{s use of steel by steel}   \\
&\quad+\text{next year}\textrm{'}\text{s use of steel by auto}  \\
&\quad+\text{next year}\textrm{'}\text{s use of steel by others} \\
\text{next year}\textrm{'}\text{s production of autos}
&=\text{next year}\textrm{'}\text{s use of autos by steel}   \\
&\quad+\text{next year}\textrm{'}\text{s use of autos by auto}  \\
&\quad+\text{next year}\textrm{'}\text{s use of autos by others}
\end{array}$$

On the left side of those equations go the unknowns $s$ and $a$. At the
ends of the right sides go our external demand estimates for next year
$17,589$ and $21,243$. For the remaining four terms, we look to the
table of this year\'s information about how the industries interact.

For instance, for next year\'s use of steel by steel, we note that this
year the steel industry used $5395$ units of steel input to produce
$25,448$ units of steel output. So next year, when the steel industry
will produce $s$ units out, we expect that doing so will take
$s\cdot (5395)/(25\,448)$ units of steel input--- this is simply the
assumption that input is proportional to output. (We are assuming that
the ratio of input to output remains constant over time; in practice,
models may try to take account of trends of change in the ratios.)

Next year\'s use of steel by the auto industry is similar. This year,
the auto industry uses $2664$ units of steel input to produce $30346$
units of auto output. So next year, when the auto industry\'s total
output is $a$, we expect it to consume $a\cdot (2664)/(30346)$ units of
steel.

Filling in the other equation in the same way, we get this system of
linear equation.

$$\begin{array}{*{3}{rc}r}
{\displaystyle\frac{5\,395}{25\,448}}\cdot s
&+ &{\displaystyle\frac{2\,664}{30\,346}}\cdot a &+ &17\,589
&= &s \\[1em]
{\displaystyle\frac{48}{25\,448}}\cdot s
&+ &{\displaystyle\frac{9\,030}{30\,346}}\cdot a &+ &21\,243
&= &a
\end{array}$$

Gauss\' method on this system.

$$\begin{array}{*{2}{rc}r}
(20\,053/25\,448)s &- &(2\,664/30\,346)a &= &17\,589 \\
-(48/25\,448)s      &+ &(21\,316/30\,346)a &= &21\,243
\end{array}$$

gives $s=25\,698$ and $a=30\,311$.

Looking back, recall that above we described why the prediction of next
year\'s totals isn\'t as simple as adding $200$ to last year\'s steel
total and subtracting $25$ from last year\'s auto total. In fact,
comparing these totals for next year to the ones given at the start for
the current year shows that, despite the drop in external demand, the
total production of the auto industry is predicted to rise. The increase
in internal demand for autos caused by steel\'s sharp rise in business
more than makes up for the loss in external demand for autos.

One of the advantages of having a mathematical model is that we can ask
\"What if \...?\" questions. For instance, we can ask \"What if the
estimates for next year\'s external demands are somewhat off?\" To try
to understand how much the model\'s predictions change in reaction to
changes in our estimates, we can try revising our estimate of next
year\'s external steel demand from $17,589$ down to $17,489$, while
keeping the assumption of next year\'s external demand for autos fixed
at $21,243$. The resulting system

$$\begin{array}{*{2}{rc}r}
(20\,053/25\,448)s &- &(2\,664/30\,346)a &= &17\,489 \\
-(48/25\,448)s      &+ &(21\,316/30\,346)a &= &21\,243
\end{array}$$

when solved gives $s=25\,571$ and $a=30\,311$. This kind of exploration
of the model is **sensitivity analysis**. We are seeing how sensitive
the predictions of our model are to the accuracy of the assumptions.

Obviously, we can consider larger models that detail the interactions
among more sectors of an economy. These models are typically solved on a
computer, using the techniques of matrix algebra that we will develop in
Chapter Three. Some examples are given in the exercises. Obviously also,
a single model does not suit every case; expert judgment is needed to
see if the assumptions underlying the model are reasonable for a
particular case. With those caveats, however, this model has proven in
practice to be a useful and accurate tool for economic analysis. For
further reading, try  and
.

## Exercises

*Hint: these systems are easiest to solve on a computer.*   `{{TextBox|1=
;Problem 3:
This table gives the numbers for the auto-steel system from
a different year, 1947 (see {{harvnb|Leontief|1951}}).
The units here are billions of 1947 dollars.

  ''used by''
  ''steel''
  ''used by''
  ''auto''
  ''used by''
  ''others''
  ''total''

   ''value of''
  ''steel''
  6.90
  1.28

  18.69

  ''value of''
  ''auto''
  0
  4.40

  14.27

 Solve for total output if next year's external
demands are: steel's demand up 10% and auto's demand up
15%.

 How do the ratios compare to those given above in the
discussion for the 1958 economy?

 Solve the 1947 equations with the 1958 external demands
(note the difference in units; a 1947 dollar buys about
what $1.30 in 1958 dollars buys).
How far off are the predictions for total output?

}}`{=mediawiki}  `{{TextBox|1=
;Problem 5:
This table gives the interrelationships among three segments of an
economy (see {{harvnb|Clark|Coupe|1967}}).

  ''used by''
  ''food''
  ''used by''
  ''wholesale''
  ''used by''
  ''retail''
  ''used by''
  ''others''
  ''total''

   ''value of''
  ''food''
  0
  2 318
  4 679

  11 869

  ''value of''
  ''wholesale''
  393
  1 089
  22 459

  122 242

  ''value of''
  ''retail''
  3
  53
  75

  116 041

We will do an Input-Output analysis on this
system.

 Fill in the numbers for this year's external
demands.

 Set up the linear system, leaving next year's
external demands blank.

 Solve the system where next year's external demands
are calculated by taking this year's external demands and
inflating them 10%.
Do all three sectors increase their total business by
10%?
Do they all even increase at the same rate?

 Solve the system where next year's external demands
are calculated by taking this year's external demands and
reducing them 7%.
(The study from which these numbers are taken concluded that
because of the closing of a local military facility, overall
personal income in the area would fall 7%, so this might
be a first guess at what would actually happen.)
/Solutions/

## References

-       .

-       .

-       .


# Linear Algebra/Input-Output Analysis M File

|current=Input-Output Analysis M File
|previous=Topic: Input-Output Analysis

``` matlab
 # Octave commands for _Linear Algebra_ by Jim Hefferon,
 # Topic: leontif.tex
 a=[(25448-5395)/25448  -2664/30346;
     -48/25448          (30346-9030)/30346];
 b=[17589;
    21243];
 ans=a \ b;
 printf("The answer to the first system is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 b=[17489;
    21243];
 ans=a \ b;
 printf("The answer to the second system is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 # question 1
 b=[17789;
    21243];
 ans=a \ b;
 printf("The answer to question (1a) is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 b=[17689;
    21443];
 ans=a \ b;
 printf("The answer to question (1b) is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 b=[17789;
    21443];
 ans=a \ b;
 printf("The answer to question (1c) is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 # question 2
 printf("Current ratio for use of steel by auto is %0.4f\n",2664/30346);
 a=[(25448-5395)/25448  -0.0500;
     -48/25448          (30346-9030)/30346];
 b=[17589;
    21243];
 ans=a \ b;
 printf("The answer to 2(a) is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 b=[17589;
    21500];
 ans=a \ b;
 printf("The answer to 2(b) is s=%0.0f and a=%0.0f\n",ans(1),ans(2));
 # question 3
 printf("The value of steel used by others is %0.2f\n",18.69-(6.90+1.28));
 printf("The value of autos used by others is %0.2f\n",14.27-(0+4.40));
 a=[(18.69-6.90)/18.69  -1.28/14.27;
     -0/18.69          (14.27-4.40)/14.27];
 b=[1.10*(18.69-(6.90+1.28));
    1.15*(14.27-(0+4.40))];
 ans=a \ b;
 printf("The answer to 3(a) is s=%0.2f and a=%0.2f\n",ans(1),ans(2));
 printf("The 1947 ratio of steel used by steel is %0.2f\n",(18.69-6.90)/18.69);
 printf("The 1947 ratio of steel used by autos is %0.2f\n",1.28/14.27);
 printf("The 1947 ratio of autos used by steel is %0.2f\n",0/18.69);
 printf("The 1947 ratio of autos used by autos is %0.2f\n",(14.27-4.40)/14.27);
 printf("The 1958 ratio of steel used by steel is %0.2f\n",(25448-5395)/25448);
 printf("The 1958 ratio of steel used by autos is %0.2f\n",2664/30346);
 printf("The 1958 ratio of autos used by steel is %0.2f\n",48/25448);
 printf("The 1958 ratio of autos used by autos is %0.2f\n",(30346-9030)/30346);
 b=[17.598/1.30;
    21.243/1.30];
 ans=a \ b;
 newans=1.30 * ans;
 printf("The answer to 3(c) is (in billions of 1947 dollars) s=%0.2f and a=%0.2f\n  and in billions of 1958 dollars it is s=%0.2f and     a=%0.2f\n",ans(1),ans(2),newans(1),newans(2));
```

|current=Input-Output Analysis M File
|previous=Topic: Input-Output Analysis
|next=Topic: Accuracy of Computations}}`{=mediawiki}


# Linear Algebra/Topic: Accuracy of Computations

|current=Topic: Accuracy of Computations
|previous=Input-Output Analysis M File
Gauss\' method lends itself nicely to computerization. The code below
illustrates. It operates on an $n \! \times \! n$ matrix `a`, pivoting
with the first row, then with the second row, etc.

``` C
for (pivot_row = 1; pivot_row <= n - 1; pivot_row++) {
    for (row_below = pivot_row + 1; row_below <= n; row_below++) {
        multiplier = a[row_below, pivot_row] / a[pivot_row, pivot_row];
        for (col = pivot_row; col <= n; col++) {
            a[row_below, col] -= multiplier * a[pivot_row, col];
        }
    }
}
```

(This code is in the C language. Here is a brief translation. The loop
construct `for (pivot_row = 1; pivot_row <= n - 1; pivot_row++) { ... }`
sets `pivot_row` to 1 and then iterates while `pivot_row` is less than
or equal to $n-1$, each time through incrementing `pivot_row` by one
with the \"`++`\" operation. The other non-obvious construct is that the
\"`-=`\" in the innermost loop amounts to the
`a[row_below, col] =- multiplier * a[pivot_row, col] + a[row_below, col]}`
operation.)

While this code provides a quick take on how Gauss\' method can be
mechanized, it is not ready to use. It is naive in many ways. The most
glaring way is that it assumes that a nonzero number is always found in
the `pivot_row, pivot_row` position for use as the pivot entry. To make
it practical, one way in which this code needs to be reworked is to
cover the case where finding a zero in that location leads to a row
swap, or to the conclusion that the matrix is singular.

Adding some `if`$\cdots$ statements to cover those cases is not hard,
but we will instead consider some more subtle ways in which the code is
naive. There are pitfalls arising from the computer\'s reliance on
finite-precision floating point arithmetic.

For example, we have seen above that we must handle as a separate case a
system that is singular. But systems that are nearly singular also
require care. Consider this one.

$$\begin{array}{*{2}{rc}r}
x &+ &2y &= &3  \\
1.000\,000\,01x &+ &2y &= &3.000\,000\,01
\end{array}$$

By eye we get the solution $x=1$ and $y=1$. But a computer has more
trouble. A computer that
represents real numbers to eight significant places (as is common,
usually called **single precision**) will represent the second equation
internally as $1.000\,000\,0x+2y=3.000\,000\,0$, losing the digits in
the ninth place. Instead of reporting the correct solution, this
computer will report something that is not even close--- this computer
thinks that the system is singular because the two equations are
represented internally as equal.

For some intuition about how the computer could come up with something
that far off, we can graph the system.

```{=html}
<center>
```
![](Linalg_singular_system.png "Linalg_singular_system.png"){width=""
height="200"}

```{=html}
</center>
```
At the scale of this graph, the two lines cannot be resolved apart. This
system is nearly singular in the sense that the two lines are nearly the
same line. Near-singularity gives this system the property that a small
change in the system can cause a large change in its solution; for
instance, changing the $3.000\,000\,01$ to $3.000\,000\,03$ changes the
intersection point from $(1,1)$ to $(3,0)$. This system changes
radically depending on a ninth digit, which explains why the eight-place
computer has trouble. A problem
that is very sensitive to inaccuracy or uncertainties in the input
values is **ill-conditioned**.

The above example gives one way in which a system can be difficult to
solve on a computer. It has the advantage that the picture of
nearly-equal lines gives a memorable insight into one way that numerical
difficulties can arise. Unfortunately this insight isn\'t very useful
when we wish to solve some large system. We cannot, typically, hope to
understand the geometry of an arbitrary large system. In addition, there
are ways that a computer\'s results may be unreliable other than that
the angle between some of the linear surfaces is quite small.

For an example, consider the system below, from
.

$$\begin{array}{*{2}{rc}r}
0.001x  &+  &y  &=  &1  \\
x  &-  &y  &=  &0
\end{array}
\qquad(*)$$

The second equation gives $x=y$, so $x=y=1/1.001$ and thus both
variables have values that are just less than $1$. A computer using two
digits represents the system internally in this way (we will do this
example in two-digit floating point arithmetic, but a similar one with
eight digits is easy to invent).

$$\begin{array}{*{2}{rc}r}
(1.0\times 10^{-3})x  &+  &(1.0\times 10^{0})y  &=  &1.0\times 10^{0}  \\
(1.0\times 10^{0})x   &-  &(1.0\times 10^{0})y  &=  &0.0\times 10^{0}
\end{array}$$

The computer\'s row reduction step $-1000\rho_1+\rho_2$ produces a
second equation $-1001y=-999$, which the computer rounds to two places
as $(-1.0\times 10^{3})y=-1.0\times 10^{3}$. Then the computer decides
from the second equation that $y=1$ and from the first equation that
$x=0$. This $y$ value is fairly good, but the $x$ is quite bad. Thus,
another cause of unreliable output is a mixture of floating point
arithmetic and a reliance on pivots that are small.

An experienced programmer may respond that we should go to **double
precision** where sixteen significant digits are retained. This will
indeed solve many problems. However, there are some difficulties with it
as a general approach. For one thing, double precision takes longer than
single precision (on a \'486 chip, multiplication takes eleven ticks in
single precision but fourteen in double precision
) and has twice the memory
requirements. So attempting to do all calculations in double precision
is just not practical. And besides, the
above systems can obviously be tweaked to give the same trouble in the
seventeenth digit, so double precision won\'t fix all problems. What we
need is a strategy to minimize the numerical trouble arising from
solving systems on a computer, and some guidance as to how far the
reported solutions can be trusted.

Mathematicians have made a careful study of how to get the most reliable
results. A basic improvement on the naive code above is to not simply
take the entry in the *pivot_row*, *pivot_row* position for the pivot,
but rather to look at all of the entries in the *pivot_row* column below
the *pivot_row* row, and take the one that is most likely to give
reliable results (e.g., take one that is not too small).
This strategy is **partial
pivoting**. For example, to solve the troublesome system ($*$) above, we
start by looking at both equations for a best first pivot, and taking
the $1$ in the second equation as more likely to give good results.
Then, the pivot step of $-.001\rho_2+\rho_1$ gives a first equation of
$1.001y=1$, which the computer will represent as
$(1.0\times 10^{0})y=1.0\times 10^{0}$, leading to the conclusion that
$y=1$ and, after back-substitution, $x=1$, both of which are close to
right. The code from above can be adapted to this purpose.

``` C
for (pivot_row = 1; pivot_row <= n - 1; pivot_row++) {
    /* Find the largest pivot in this column (in row max). */
    max = pivot_row;
    for (row_below = pivot_row + 1; pivot_row <= n; row_below++) {
        if (abs(a[row_below, pivot_row]) > abs(a[max, row_below]))
            max = row_below;
     }

    /* Swap rows to move that pivot entry up. */
    for (col = pivot_row; col <= n; col++) {
        temp = a[pivot_row, col];
        a[pivot_row, col] = a[max, col];
        a[max, col] = temp;
     }

     /* Proceed as before. */
     for (row_below = pivot_row + 1; row_below <= n; row_below++) {
         multiplier = a[row_below, pivot_row] / a[pivot_row, pivot_row];
         for (col = pivot_row; col <= n; col++) {
             a[row_below, col] -= multiplier * a[pivot_row, col];
         }
     }
}
```

A full analysis of the best way to implement Gauss\' method is outside
the scope of the book (see ), but
the method recommended by most experts is a variation on the code above
that first finds the best pivot among the candidates, and then scales it
to a number that is less likely to give trouble.
This is **scaled partial
pivoting**.

In addition to returning a result that is likely to be reliable, most
well-done code will return a number, called the **condition number**
that describes the factor by which uncertainties in the input numbers
could be magnified to become inaccuracies in the results returned (see
).

The lesson of this discussion is that just because Gauss\' method always
works in theory, and just because computer code correctly implements
that method, and just because the answer appears on green-bar paper,
doesn\'t mean that the answer is reliable. In practice, always use a
package where experts have worked hard to counter what can go wrong.

## Exercises

## References

-       .

-       .

-       .

-       .


# Linear Algebra/Topic: Analyzing Networks

|current=Topic: Analyzing Networks
|previous=Topic: Accuracy of Computations
The diagram below shows some of a car\'s electrical network. The battery
is on the left, drawn as stacked line segments. The wires are drawn as
lines, shown straight and with sharp right angles for neatness. Each
light is a circle enclosing a loop.

```{=html}
<center>
```
![](Linalg_car_circuit.png "Linalg_car_circuit.png"){width=""
height="200"}

```{=html}
</center>
```
The designer of such a network needs to answer questions like: How much
electricity flows when both the hi-beam headlights and the brake lights
are on? Below, we will use linear systems to analyze simpler versions of
electrical networks.

For the analysis we need two facts about electricity and two facts about
electrical networks.

The first fact about electricity is that a battery is like a pump: it
provides a force impelling the electricity to flow through the circuits
connecting the battery\'s ends, if there are any such circuits.
We say that the battery provides a
**potential** to flow. Of course, this network accomplishes its function
when, as the electricity flows through a circuit, it goes through a
light. For instance, when the driver steps on the brake then the switch
makes contact and a circuit is formed on the left side of the diagram,
and the electrical current flowing through that circuit will make the
brake lights go on, warning drivers behind.

The second electrical fact is that in some kinds of network components
the amount of flow is proportional to the force provided by the battery.
That is, for each such component
there is a number, its **resistance**, such that the potential is equal
to the flow times the resistance. The units of measurement are:
potential is described in **volts**, the rate of flow is in **amperes**,
and resistance to the flow is in **ohms**. These units are defined so
that $\mbox{volts}=\mbox{amperes}\cdot\mbox{ohms}$.

Components with this property, that the voltage-amperage response curve
is a line through the origin, are called **resistors**. (Light bulbs
such as the ones shown above are not this kind of component, because
their ohmage changes as they heat up.) For example, if a resistor
measures $2$ ohms then wiring it to a $12$ volt battery results in a
flow of $6$ amperes. Conversely, if we have electrical current of $2$
amperes through it then there must be a $4$ volt potential difference
between its ends. This is the
**voltage drop** across the resistor. One way to think of a electrical
circuits like the one above is that the battery provides a voltage rise
while the other components are voltage drops.

The two facts that we need about networks are Kirchhoff\'s Laws.

-   *Current Law.* For any point in a network, the flow in equals the
    flow out.
-   *Voltage Law.* Around any circuit the total drop equals the total
    rise.

In the above network there is only one voltage rise, at the battery, but
some networks have more than one.

For a start we can consider the network below. It has a battery that
provides the potential to flow and three resistors (resistors are drawn
as zig-zags). When components are wired
one after another, as here, they are said to be in **series**.

```{=html}
<center>
```
![](Linalg_resisters_in_series.png "Linalg_resisters_in_series.png"){width=""
height="200"}

```{=html}
</center>
```
By Kirchhoff\'s Voltage Law, because the voltage rise is $20$ volts, the
total voltage drop must also be $20$ volts. Since the resistance from
start to finish is $10$ ohms (the resistance of the wires is
negligible), we get that the current is $(20/10)=2$ amperes. Now, by
Kirchhoff\'s Current Law, there are $2$ amperes through each resistor.
(And therefore the voltage drops are: $4$ volts across the $2$ oh m
resistor, $10$ volts across the $5$ ohm resistor, and $6$ volts across
the $3$ ohm resistor.)

The prior network is so simple that we didn\'t use a linear system, but
the next network is more complicated.
In this one, the resistors are in
**parallel**. This network is more like the car lighting diagram shown
earlier.

```{=html}
<center>
```
![](Linalg_resisters_in_parallel.png "Linalg_resisters_in_parallel.png"){width=""
height="200"}

```{=html}
</center>
```
We begin by labeling the branches, shown below. Let the current through
the left branch of the parallel portion be $i_1$ and that through the
right branch be $i_2$, and also let the current through the battery be
$i_0$. (We are following Kirchoff\'s Current Law; for instance, all
points in the right branch have the same current, which we call $i_2$.
Note that we don\'t need to know the actual direction of flow--- if
current flows in the direction opposite to our arrow then we will simply
get a negative number in the solution.)

```{=html}
<center>
```
![](Linalg_resisters_in_parallel_2.png "Linalg_resisters_in_parallel_2.png"){width=""
height="200"}

```{=html}
</center>
```
The Current Law, applied to the point in the upper right where the flow
$i_0$ meets $i_1$ and $i_2$, gives that $i_0=i_1+i_2$. Applied to the
lower right it gives $i_1+i_2=i_0$. In the circuit that loops out of the
top of the battery, down the left branch of the parallel portion, and
back into the bottom of the battery, the voltage rise is $20$ while the
voltage drop is $i_1\cdot 12$, so the Voltage Law gives that $12i_1=20$.
Similarly, the circuit from the battery to the right branch and back to
the battery gives that $8i_2=20$. And, in the circuit that simply loops
around in the left and right branches of the parallel portion
(arbitrarily taken clockwise), there is a voltage rise of $0$ and a
voltage drop of $8i_2-12i_1$ so the Voltage Law gives that
$8i_2-12i_1=0$.

$$\begin{array}{*{3}{rc}r}
i_0&- &i_1    &-  &i_2   &=  &0 \\
-i_0&+ &i_1    &+  &i_2   &=  &0  \\
&  &12i_1  &   &      &=  &20  \\
&  &       &   &8i_2  &=  &20  \\
&  &-12i_1 &+  &8i_2  &=  &0
\end{array}$$

The solution is $i_0=25/6$, $i_1=5/3$, and $i_2=5/2$, all in amperes.
(Incidentally, this illustrates that redundant equations do indeed arise
in practice.)

Kirchhoff\'s laws can be used to establish the electrical properties of
networks of great complexity.
The next diagram shows five
resistors, wired in a **series-parallel** way.

```{=html}
<center>
```
![](Linalg_wheatstone_bridge.png "Linalg_wheatstone_bridge.png"){width=""
height="200"}

```{=html}
</center>
```
This network is a **Wheatstone bridge** (see Problem 4 ). To analyze it,
we can place the arrows in this way.

```{=html}
<center>
```
![](Linalg_wheatstone_bridge_2.png "Linalg_wheatstone_bridge_2.png"){width=""
height="200"}

```{=html}
</center>
```
Kirchoff\'s Current Law, applied to the top node, the left node, the
right node, and the bottom node gives these.

$$\begin{array}{rl}
i_0     &=  i_1+i_2  \\
i_1     &=  i_3+i_5  \\
i_2+i_5 &=  i_4      \\
i_3+i_4 &=  i_0
\end{array}$$

Kirchhoff\'s Voltage Law, applied to the inside loop (the $i_0$ to $i_1$
to $i_3$ to $i_0$ loop), the outside loop, and the upper loop not
involving the battery, gives these.

$$\begin{array}{rl}
5i_1+10i_3  &= 10   \\
2i_2+4i_4   &= 10   \\
5i_1+50i_5-2i_2  &= 0
\end{array}$$

Those suffice to determine the solution $i_0=7/3$, $i_1=2/3$, $i_2=5/3$,
$i_3=2/3$, $i_4=5/3$, and $i_5=0$.

Networks of other kinds, not just electrical ones, can also be analyzed
in this way. For instance, networks of streets are given in the
exercises.

## Exercises

*Many of the systems for these problems are mostly easily solved on a
computer.*

*There are networks other than electrical ones, and we can ask how well
Kirchoff\'s laws apply to them. The remaining questions consider an
extension to networks of streets.*


# Linear Algebra/Topic: Speed of Gauss' Method

|current=Topic: Speed of Gauss' Method
|previous=Topic: Analyzing Networks
Gauss\' Method to solve the linear systems in this book because it is
easy to understand, easily shown to give the right answers, and fast. It
is fast in that, in all the by-hand calculations we have needed, we have
gotten the answers in only a few steps, taking only a few minutes.
However, scientists and engineers who solve linear systems in practice
must have a method that is fast enough for large systems, with 1000
equations or 10,000 equations or even 100,000 equations. These systems
are solved on a computer, so the speed of the machine helps, but
nonetheless the speed of the method used is a major consideration, and
is sometimes the factor limiting which problems can be solved.

The speed of an algorithm is usually measured by finding how the time
taken to solve problems grows as the size of the input data set grows.
That is, how much longer will the algorithm take if we increase the size
of the input data by a factor of ten, say from a 1000-equation system to
a 10,000-equation system, or from 10,000 to 100,000? Does the time taken
grow ten times, or a hundred times, or a thousand times? Is the time
taken by the algorithm proportional to the size of the data set, or to
the square of that size, or to the cube of that size, etc.?

Here is a fragment of Gauss\' Method code, implemented in the computer
language FORTRAN. The coefficients of the linear system are stored in
the $N \! \times \! N$ array *A*, and the constants are stored in the
$N \! \times \! 1$ array *B*. For each *ROW* between $1$ and $N$ this
program has already found the pivot entry $A(ROW,COL)$. Now it will
pivot.

$$-PIVINV\cdot \rho_{ROW}
+\rho_{i}$$

(This code fragment is for illustration only, and is incomplete. For
example, see the later topic on the Accuracy of Gauss\' Method.
Nonetheless, this fragment will do for our purposes because analysis of
finished versions, including all the tests and sub-cases, is messier but
gives essentially the same result.)

``` fortran
PIVINV=1./A(ROW,COL)
DO 10 I=ROW+1, N
DO 20 J=I, N
A(I,J)=A(I,J)-PIVINV*A(ROW,J)
20 CONTINUE
B(J)=B(J)-PIVINV*B(ROW)
10 CONTINUE
```

The outermost loop (not shown) runs through $N-1$ rows. For each of
these rows, the shown loops perform arithmetic on the entries in *A*
that are below and to the right of the pivot entry (and also on the
entries in *B*, but to simplify the analysis we will not count those
operations\-\--see Exercise ). We will assume the pivot is found in the
usual place, that is, that $COL=ROW$ (as above, analysis of the general
case is messier but gives essentially the same result). That means there
are $(N-ROW)^2$ entries to perform arithmetic on. On average, ROW will
be $N/2$. Thus we estimate the nested loops above will run something
like $(N/2)^2$ times, that is, will run in a time proportional to the
square of the number of equations. Taking into account the outer loop
that is not shown, we get the estimate that the running time of the
algorithm is proportional to the cube of the number of equations.

Algorithms that run in time directly proportional to the size of the
data set are fast, algorithms that run in time proportional to the
square of the size of the data set are less fast, but typically quite
usable, and algorithms that run in time proportional to the cube of the
size of the data set are still reasonable in speed.

Speed estimates like these are a good way of understanding how quickly
or slowly an algorithm can be expected to run on average. There are
special cases, however, of systems on which the above Gauss\' method
code is especially fast, so there may be factors about a problem that
make it especially suitable for this kind of solution.

In practice, the code found in computer algebra systems, or in the
standard packages, implements a variant on Gauss\' method, called
triangular factorization. To state this method requires the language of
matrix algebra, which we will not see until Chapter Three. Nonetheless,
the above code is conceptually quite close to that usually used in
applications.

There have been some theoretical speed-ups in the running time required
to solve linear systems. Algorithms other than Gauss\' method have been
invented that take a time proportional not to the cube of the size of
the data set, but instead to the (approximately) $2.7$ power (this is
still under active research, so this exponent may come down somewhat
over time). However, these theoretical improvements have not come into
widespread use, in part because the new methods take a quite large data
set before they overtake Gauss\' method (although they will outperform
Gauss\' method on very large sets, there is some startup overhead that
keeps them from being faster on the systems that have, so far, been
solved in practice).

## Exercises


# Linear Algebra/Appendix

|current=Appendix
|previous=Topic: Linear Recurrences
made of arguments (reasoned discourse that is, not crockery-throwing).
This section is a reference to the most used techniques. A reader having
trouble with, say, proof by contradiction, can turn here for an outline
of that method.

But this section gives only a sketch. For more, these are classics:
*Methods of Logic* by Quine, *Induction and Analogy in Mathematics* by
Pólya, and *Naive Set Theory* by Halmos. Reader can also read the
wikibook Mathematical Proof.
|current=Appendix
|previous=Topic: Linear Recurrences
|next=Propositions}}`{=mediawiki}


# Linear Algebra/Propositions

|current=Propositions
|previous=Appendix
The point at issue in an argument is the
**proposition**. Mathematicians usually
write the point in full before the proof and label it either **Theorem**
for major points, **Corollary** for points that follow immediately from
a prior one, or **Lemma** for results chiefly used to prove other
results.

The statements expressing propositions can be complex, with many
subparts. The truth or falsity of the entire proposition depends both on
the truth value of the parts, and on the words used to assemble the
statement from its parts.

### Not

For example, where $P$ is a proposition, \"it is not the case that $P$\"
is true provided that $P$ is false. Thus, \"$n$ is not prime\" is true
only when $n$ is the product of smaller integers.

We can picture the \"not\" operation with a **Venn diagram**.

```{=html}
<center>
```
![](Linalg_venn_not.png "Linalg_venn_not.png"){width="" height="200"}

```{=html}
</center>
```
Where the box encloses all natural numbers, and inside the circle are
the primes, the shaded area holds numbers satisfying \"not $P$\".

To prove that a \"not $P$\" statement holds, show that $P$ is false.

### And

Consider the statement form \"$P$ and $Q$\". For the statement to be
true both halves must hold: \"$7$ is prime and so is $3$\" is true,
while \"$7$ is prime and $3$ is not\" is false.

Here is the Venn diagram for \"$P$ and $Q$\".

```{=html}
<center>
```
![](Linalg_venn_and.png "Linalg_venn_and.png"){width="" height="200"}

```{=html}
</center>
```
To prove \"$P$ and $Q$\", prove that each half holds.

### Or

A \"$P$ or $Q$\" is true when either half holds: \"$7$ is prime or $4$
is prime\" is true, while \"$7$ is not prime or $4$ is prime\" is false.
We take \"or\" inclusively so that if both halves are true \"$7$ is
prime or $4$ is not\" then the statement as a whole is true. (In
everyday speech, sometimes \"or\" is meant in an exclusive way--- \"Eat
your vegetables or no dessert\" does not intend both halves to hold---
but we will not use \"or\" in that way.)

The Venn diagram for \"or\" includes all of both circles.

```{=html}
<center>
```
![](Linalg_venn_and.png "Linalg_venn_and.png"){width="" height="200"}

```{=html}
</center>
```
To prove \"$P$ or $Q$\", show that in all cases at least one half holds
(perhaps sometimes one half and sometimes the other, but always at least
one).

### If-then

An \"if $P$ then $Q$\" statement (sometimes written \"$P$ materially
implies $Q$\" or just \"$P$ implies $Q$\" or \"$P\implies Q$\") is true
unless $P$ is true while $Q$ is false. Thus \"if $7$ is prime then $4$
is not\" is true while \"if $7$ is prime then $4$ is also prime\" is
false. (Contrary to its use in casual speech, in mathematics \"if $P$
then $Q$\" does not connote that $P$ precedes $Q$ or causes $Q$.)

More subtly, in mathematics \"if $P$ then $Q$\" is true when $P$ is
false: \"if $4$ is prime then $7$ is prime\" and \"if $4$ is prime then
$7$ is not\" are both true statements, sometimes said to be **vacuously
true**. We adopt this convention because we want statements like \"if a
number is a perfect square then it is not prime\" to be true, for
instance when the number is $5$ or when the number is $6$.

The diagram

```{=html}
<center>
```
![](Linalg_venn_ifthen.png "Linalg_venn_ifthen.png"){width=""
height="200"}

```{=html}
</center>
```
shows that $Q$ holds whenever $P$ does (another phrasing is \"$P$ is
sufficient to give $Q$\"). Notice again that if $P$ does not hold, $Q$
may or may not be in force.

There are two main ways to establish an implication. The first way is
direct: assume that $P$ is true and, using that assumption, prove $Q$.
For instance, to show \"if a number is divisible by 5 then twice that
number is divisible by 10\", assume that the number is $5n$ and deduce
that $2(5n)=10n$. The second way
is indirect: prove the **contrapositive** statement: \"if $Q$ is false
then $P$ is false\" (rephrased, \"$Q$ can only be false when $P$ is also
false\"). As an example, to show \"if a number is prime then it is not a
perfect square\", argue that if it were a square $p=n^2$ then it could
be factored $p=n\cdot n$ where $n<p$ and so wouldn\'t be prime (of
course $p=0$ or $p=1$ don\'t give $n<p$ but they are nonprime by
definition).

Note two things about this statement form.

First, an \"if $P$ then $Q$\" result can sometimes be improved by
weakening $P$ or strengthening $Q$. Thus, \"if a number is divisible by
$p^2$ then its square is also divisible by $p^2$\" could be upgraded
either by relaxing its hypothesis: \"if a number is divisible by $p$
then its square is divisible by $p^2$\", or by tightening its
conclusion: \"if a number is divisible by $p^2$ then its square is
divisible by $p^4$\".

Second, after showing \"if $P$ then $Q$\", a good next step is to look
into whether there are cases where $Q$ holds but $P$ does not. The idea
is to better understand the relationship between $P$ and $Q$, with an
eye toward strengthening the proposition.

### Equivalence

An if-then statement cannot be improved when not only does $P$ imply
$Q$, but also $Q$ implies $P$. Some ways to say this are: \"$P$ if and
only if $Q$\", \"$P$ iff $Q$\", \"$P$ and $Q$ are logically
equivalent\", \"$P$ is necessary and sufficient to give $Q$\",
\"$P\iff Q$\". For example, \"a number is divisible by a prime if and
only if that number squared is divisible by the prime squared\".

The picture here shows that $P$ and $Q$ hold in exactly the same cases.

```{=html}
<center>
```
![](Linalg_venn_equiv.png "Linalg_venn_equiv.png"){width=""
height="200"}

```{=html}
</center>
```
Although in simple arguments a chain like \"$P$ if and only if $R$,
which holds if and only if $S$ \...\" may be practical, typically we
show equivalence by showing the \"if $P$ then $Q$\" and \"if $Q$ then


# Linear Algebra/Quantifiers

|current=Quantifiers
|previous=Propositions

Compare these two statements about natural numbers: \"there is an $x$
such that $x$ is divisible by $x^2$\" is true, while \"for all numbers
$x$, that $x$ is divisible by $x^2$\" is false.
We call the \"there is\" and \"for all\"
prefixes **quantifiers**.

### For all

The \"for all\" prefix is the **universal quantifier**, symbolized
$\forall$.

Venn diagrams aren\'t very helpful with quantifiers, but in a sense the
box we draw to border the diagram shows the universal quantifier since
it delineates the universe of possible members.

```{=html}
<center>
```
![](Linalg_venn_forall.png "Linalg_venn_forall.png"){width=""
height="200"}

```{=html}
</center>
```
To prove that a statement holds in all cases, we must show that it holds
in each case. Thus, to prove \"every number divisible by $p$ has its
square divisible by $p^2$\", take a single number of the form $pn$ and
square it $(pn)^2=p^2n^2$. This is a \"typical element\" or \"generic
element\" proof.

This kind of argument requires that we are careful to not assume
properties for that element other than those in the hypothesis--- for
instance, this type of wrong argument is a common mistake: \"if $n$ is
divisible by a prime, say $2$, so that $n=2k$ then $n^2=(2k)^2=4k^2$ and
the square of the number is divisible by the square of the prime\". That
is an argument about the case $p=2$, but it isn\'t a proof for general
$p$.

### There exists

We will also use the **existential quantifier**, symbolized $\exists$
and read \"there exists\".

As noted above, Venn diagrams are not much help with quantifiers, but a
picture of \"there is a number such that $P$\" would show both that
there can be more than one and that not all numbers need satisfy $P$.

```{=html}
<center>
```
![](Linalg_venn_thereexists.png "Linalg_venn_thereexists.png"){width=""
height="200"}

```{=html}
</center>
```
An existence proposition can be proved by producing something satisfying
the property: once, to settle the question of primality of $2^{2^5}+1$,
Euler produced its divisor $641$. But there are proofs showing that
something exists without saying how to find it; Euclid\'s argument given
in the next subsection shows there are infinitely many primes without
naming them. In general, while demonstrating existence is better than
nothing, giving an example is better, and an exhaustive list of all
instances is great. Still, mathematicians take what they can get.

Finally, along with \"Are there any?\" we often ask \"How many?\" That
is why the issue of uniqueness often arises in conjunction with
questions of existence. Many times the two arguments are simpler if
separated, so note that just as proving something exists does not show
it is unique, neither does proving something is unique show that it
exists. (Obviously \"the natural number with more factors than any
other\" would be unique, but in fact no such number exists.)
|current=Quantifiers
|previous=Propositions
|next=Techniques of Proof}}`{=mediawiki}


# Linear Algebra/Techniques of Proof

|current=Techniques of Proof
|previous=Quantifiers

### Induction

Many proofs are iterative, \"Here\'s why the statement is true for for
the case of the number $1$, it then follows for $2$, and from there to
$3$, and so on \...\". These are called proofs by **induction**. Such a
proof has two steps. In the **base
step** the proposition is established for some first number, often $0$
or $1$. Then in the **inductive
step** we assume that the proposition holds for numbers up to some $k$
and deduce that it then holds for the next number $k+1$.

Here is an example.

> We will prove that $1+2+3+\dots+n=n(n+1)/2$.
>
> For the base step we must show that the formula holds when $n=1$.
> That\'s easy, the sum of the first $1$ number does indeed equal
> $1(1+1)/2$.
>
> For the inductive step, assume that the formula holds for the numbers
> $1,2,\ldots,k$. That is, assume all of these instances of the formula.
>
> $$\begin{array}{rl}
> 1
> &=1(1+1)/2  \\
> \text{and}\quad 1+2
> &=2(2+1)/2  \\
> \text{and}\quad  1+2+3
> &=3(3+1)/2  \\
> &\vdots    \\
> \text{and}\quad 1+\dots+k
> &=k(k+1)/2
> \end{array}$$
>
> From this assumption we will deduce that the formula therefore also
> holds in the $k+1$ next case. The deduction is straightforward
> algebra.
>
> $$1+2+\cdots+k+(k+1)
> =
> \frac{k(k+1)}{2}+(k+1)
> =
> \frac{(k+1)(k+2)}{2}$$

We\'ve shown in the base case that the above proposition holds for $1$.
We\'ve shown in the inductive step that if it holds for the case of $1$
then it also holds for $2$; therefore it does hold for $2$. We\'ve also
shown in the inductive step that if the statement holds for the cases of
$1$ and $2$ then it also holds for the next case $3$, etc. Thus it holds
for any natural number greater than or equal to $1$.

Here is another example.

> We will prove that every integer greater than $1$ is a product of
> primes.
>
> The base step is easy: $2$ is the product of a single prime.
>
> For the inductive step assume that each of $2, 3,\ldots ,k$ is a
> product of primes, aiming to show $k+1$ is also a product of primes.
> There are two possibilities: (i) if $k+1$ is not divisible by a number
> smaller than itself then it is a prime and so is the product of
> primes, and (ii) if $k+1$ is divisible then its factors can be written
> as a product of primes (by the inductive hypothesis) and so $k+1$ can
> be rewritten as a product of primes. That ends the proof.
>
> (*Remark.* The Prime Factorization Theorem of Number Theory says that
> not only does a factorization exist, but that it is unique. We\'ve
> shown the easy half.)

There are two things to note about the \"next number\" in an induction
argument.

For one thing, while induction works on the integers, it\'s no good on
the reals. There is no \"next\" real.

The other thing is that we sometimes use induction to go down, say, from
$10$ to $9$ to $8$, etc., down to $0$. So \"next number\" could mean
\"next lowest number\". Of course, at the end we have not shown the fact
for all natural numbers, only for those less than or equal to $10$.

### Contradiction

Another technique of proof is to show something is true by showing it
can\'t be false.

> The classic example is Euclid\'s, that there are infinitely many
> primes.
>
> Suppose there are only finitely many primes $p_1,\dots,p_k$. Consider
> $p_1\cdot p_2\dots p_k +1$. None of the primes on this supposedly
> exhaustive list divides that number evenly, each leaves a remainder of
> $1$. But every number is a product of primes so this can\'t be. Thus
> there cannot be only finitely many primes.

Every proof by contradiction has the same form: **assume** that **the
false** proposition **is** **true** and **derive** some
**contradiction** to known facts. This kind of logic is known as
Aristotelian Logic, or Term Logic

> Another example is this proof that $\sqrt{2}$ is not a rational
> number.
>
> Suppose that $\sqrt{2}=m/n$.
>
> $$2n^2=m^2$$
>
> Factor out the $2$\'s: $n=2^{k_n}\cdot \hat{n}$ and
> $m=2^{k_m}\cdot \hat{m}$ and rewrite.
>
> $$2\cdot (2^{k_n}\cdot \hat{n})^2
> =
> (2^{k_m}\cdot \hat{m})^2$$
>
> The Prime Factorization Theorem says that there must be the same
> number of factors of $2$ on both sides, but there are an odd number
> $1+2k_n$ on the left and an even number $2k_m$ on the right. That\'s a
> contradiction, so a rational with a square of $2$ cannot be.

Both of these examples aimed to prove something doesn\'t exist. A
negative proposition often suggests a proof by contradiction.
|current=Techniques of Proof
|previous=Quantifiers
|next=Sets, Functions, Relations}}`{=mediawiki}


# Linear Algebra/Sets, Functions, Relations

|current=Sets, Functions, Relations
|previous=Techniques of Proof

### Sets

Mathematicians work with collections called **sets**. A set can be given
as a listing between curly braces as in $\{ 1,4,9,16 \}$, or, if that\'s
unwieldy, by using set-builder notation as in
$\{x\,\big|\, x^5-3x^3+2=0 \}$ (read \"the set of all $x$ such that
\\ldots\"). We name sets with capital roman letters as with the primes
$P=\{2,3,5,7,11,\ldots\,\}$, except for a few special sets such as the
real numbers $\mathbb{R}$, and the complex numbers $\mathbb{C}$.
To denote that something is an
**element** (or **member**) of a set we use \"${}\in {}$\", so that
$7\in\{3,5,7\}$ while $8\not\in\{3,5,7\}$.

What distinguishes a set from any other type of collection is the
Principle of Extensionality, that two sets with the same elements are
equal. Because of this principle, in a set repeats collapse
$\{7,7\}=\{7\}$ and order doesn\'t matter $\{2,\pi\}=\{\pi,2\}$.

We use \"$\subset$\" for the subset relationship:
$\{2,\pi\}\subset\{2,\pi,7\}$ and \"$\subseteq$\" for subset or equality
(if $A$ is a subset of $B$ but $A\neq
B$ then $A$ is a **proper subset** of $B$). These symbols may be
flipped, for instance $\{ 2,\pi,5\}\supset\{2,5\}$.

Because of Extensionality, to prove that two sets are equal $A=B$, just
show that they have the same members.
Usually we show mutual
inclusion, that both $A\subseteq B$ and $A\supseteq B$.

### Set operations

Venn diagrams are handy here. For instance, $x\in P$ can be pictured

```{=html}
<center>
```
![](Linalg_venn_xinP.png "Linalg_venn_xinP.png"){width="" height="200"}

```{=html}
</center>
```
and \"$P\subseteq Q$\" looks like this.

```{=html}
<center>
```
![](Linalg_venn_ifthen.png "Linalg_venn_ifthen.png"){width=""
height="200"}

```{=html}
</center>
```
Note that this is a repeat of the diagram for \"if \\ldots then \...\"
propositions. That\'s because \"$P\subseteq Q$\" means \"if $x\in P$
then $x\in Q$\".

In general, for every propositional logic operator there is an
associated set operator. For
instance, the **complement** of $P$ is
$P^{\text{comp}}=\{x\,\big|\, \text{not }(x\in P)\}$

```{=html}
<center>
```
![](Linalg_venn_not.png "Linalg_venn_not.png"){width="" height="200"}

```{=html}
</center>
```
the **union** is $P\cup Q=\{x\,\big|\,(x\in P) \text{ or }(x\in Q)\}$

```{=html}
<center>
```
![](Linalg_venn_or.png "Linalg_venn_or.png"){width="" height="200"}

```{=html}
</center>
```
and the **intersection** is
$P\cap Q=\{x\,\big|\, (x\in P)\text{ and }(x\in Q)\}.$

```{=html}
<center>
```
![](Linalg_venn_and.png "Linalg_venn_and.png"){width="" height="200"}

```{=html}
</center>
```
}}When two sets share no members their intersection is the **empty set**
$\{\}$, symbolized $\varnothing$. Any set has the empty set for a
subset, by the \"vacuously true\" property of the definition of
implication.

### Sequences

We shall also use collections where order does matter and where repeats
do not collapse. These are **sequences**, denoted with angle brackets:
$\langle  2,3,7 \rangle \neq\langle 2,7,3 \rangle$.
A sequence of length $2$ is
sometimes called an **ordered pair** and written with parentheses:
$(\pi,3)$. We also sometimes say \"ordered triple\", \"ordered
$4$-tuple\", etc. The set of ordered $n$-tuples of elements of a set $A$
is denoted $A^n$. Thus the set of pairs of reals is $\mathbb{R}^2$.

### Functions

We first see functions in elementary Algebra, where they are presented
as formulas (e.g., $f(x)=16x^2-100$), but progressing to more advanced
Mathematics reveals more general functions--- trigonometric ones,
exponential and logarithmic ones, and even constructs like absolute
value that involve piecing together parts--- and we see that functions
aren\'t formulas, instead the key idea is that a function associates
with its input $x$ a single output $f(x)$.

Consequently, a **function** or **map** is defined to be a set of
ordered pairs $(x,f(x)\,)$ such that $x$ suffices to determine $f(x)$,
that is: if $x_1=x_2$ then $f(x_1)=f(x_2)$ (this requirement is referred
to by saying a function is **well-defined**).\\footnote{More on this is
in the section on isomorphisms}

Each input $x$ is one of the function\'s **arguments** and each output
$f(x)$ is a **value**. The set of all
arguments is $f$\'s **domain** and the set of output values is its
**range**. Usually we don\'t need know
what is and is not in the range and we instead work with a superset of
the range, the **codomain**. The notation for a function $f$ with domain
$X$ and codomain $Y$ is $f:X\to Y$.

```{=html}
<center>
```
![](Linalg_domain_range_codomain.png "Linalg_domain_range_codomain.png"){width=""
height="200"}

```{=html}
</center>
```
We sometimes instead use the notation
$x\stackrel{f}{\longmapsto} 16x^2-100$, read \"$x$ maps under $f$ to
$16x^2-100$\", or \"$16x^2-100$ is the \"*image*\' of $x$\'.

Some maps, like $x\mapsto \sin(1/x)$, can be thought of as combinations
of simple maps, here, $g(y)=\sin(y)$ applied to the image of $f(x)=1/x$.
The **composition** of $g:Y\to Z$
with $f:X\to Y$, is the map sending $x\in X$ to $g(\, f(x)\,)\in Z$. It
is denoted $g\circ f:X\to Z$. This definition only makes sense if the
range of $f$ is a subset of the domain of $g$.

Observe that the **identity map** $\mbox{id}:Y\to Y$ defined by
$\mbox{id}(y)=y$ has the property that for any $f:X\to Y$, the
composition $\mbox{id}\circ f$ is equal to $f$. So an identity map plays
the same role with respect to function composition that the number $0$
plays in real number addition, or that the number $1$ plays in
multiplication.

In line with that analogy, define a **left inverse** of a map $f:X\to Y$
to be a function $g:\text{range}(f)\to X$ such that $g\circ f$ is the
identity map on $X$. Of course, a
**right inverse** of $f$ is a $h:Y\to X$ such that $f\circ h$ is the
identity.

A map that is both a left and right inverse of $f$ is called simply an
**inverse**. An inverse, if one exists, is unique because if both $g_1$
and $g_2$ are inverses of $f$ then $g_1(x)=g_1\circ  (f\circ g_2) (x)
= (g_1\circ f) \circ g_2(x)
=g_2(x)$ (the middle equality comes from the associativity of function
composition), so we often call it \"the\" inverse, written $f^{-1}$. For
instance, the inverse of the function $f:\mathbb{R}\to \mathbb{R}$ given
by $f(x)=2x-3$ is the function $f^{-1}:\mathbb{R}\to \mathbb{R}$ given
by $f^{-1}(x)=(x+3)/2$.

The superscript \"$f^{-1}$\" notation for function inverse can be
confusing--- it doesn\'t mean $1/f(x)$. It is used because it fits into
a larger scheme. Functions that have the same codomain as domain can be
iterated, so that where $f:X\to X$, we can consider the composition of
$f$ with itself: $f\circ f$, and $f\circ f\circ f$, etc.

Naturally enough, we write $f\circ f$ as $f^2$ and $f\circ f\circ f$ as
$f^3$, etc. Note that the familiar exponent rules for real numbers
obviously hold: $f^i\circ f^j=f^{i+j}$ and $(f^i)^j=f^{i\cdot j}$. The
relationship with the prior paragraph is that, where $f$ is invertible,
writing $f^{-1}$ for the inverse and $f^{-2}$ for the inverse of $f^2$,
etc., gives that these familiar exponent rules continue to hold, once
$f^0$ is defined to be the identity map.

If the codomain $Y$ equals the range of $f$ then we say that the
function is **onto** (or **surjective**). A function has a right inverse
if and only if it is onto (this is not hard to check).
If no two arguments share an image, if
$x_1\neq x_2$ implies that $f(x_1)\neq f(x_2)$, then the function is
**one-to-one** (or **injective**). A function has a left inverse if and
only if it is one-to-one (this is also not hard to check).

By the prior paragraph, a map has an inverse if and only if it is both
onto and one-to-one; such a function is a **correspondence**. It
associates one and only one element of the domain with each element of
the range (for example, finite sets must have the same number of
elements to be matched up in this way). Because a composition of
one-to-one maps is one-to-one, and a composition of onto maps is onto, a
composition of correspondences is a correspondence.

We sometimes want to shrink the domain of a function. For instance, we
may take the function $f:\mathbb{R}\to \mathbb{R}$ given by $f(x)=x^2$
and, in order to have an inverse, limit input arguments to nonnegative
reals $\hat{f}:\mathbb{R}^+\to \mathbb{R}$.
Technically, $\hat{f}$ is a
different function than $f$; we call it the **restriction** of $f$ to
the smaller domain.

A final point on functions: neither $x$ nor $f(x)$ need be a number. As
an example, we can think of $f(x,y)=x+y$ as a function that takes the
ordered pair $(x,y)$ as its argument.

### Relations

Some familiar operations are obviously functions: addition maps $(5,3)$
to $8$. But what of \"$<$\" or \"$=$\"? We here take the approach of
rephrasing \"$3<5$\" to \"$(3,5)$ is in the relation $<$\".
That is, define a **binary relation** on
a set $A$ to be a set of ordered pairs of elements of $A$. For example,
the $<$ relation is the set $\{(a,b)\,\big|\, a<b\}$; some elements of
that set are $(3,5)$, $(3,7)$, and $(1,100)$.

Another binary relation on the natural numbers is equality; this
relation is formally written as the set
$\{\ldots,(-1,-1),(0,0),(1,1),\ldots\}$.

Still another example is \"closer than $10$\", the set
$\{(x,y)\,\big|\, |x-y|<10 \}$. Some members of that relation are
$(1,10)$, $(10,1)$, and $(42,44)$. Neither $(11,1)$ nor $(1,11)$ is a
member.

Those examples illustrate the generality of the definition. All kinds of
relationships (e.g., \"both numbers even\" or \"first number is the
second with the digits reversed\") are covered under the definition.

### Equivalence Relations

We shall need to say, formally, that two objects are alike in some way.
While these alike things aren\'t identical, they are related (e.g., two
integers that \"give the same remainder when divided by $2$\").

A binary relation $\{(a,b),\ldots \}$ is an **equivalence relation**when
it satisfies

1.  **reflexivity**: any object is
    related to itself;
2.  **symmetry**: if $a$ is related to
    $b$ then $b$ is related to $a$;
3.  **transitivity**: if $a$ is
    related to $b$ and $b$ is related to $c$ then $a$ is related to $c$.

(To see that these conditions formalize being the same, read them again,
replacing \"is related to\" with \"is like\".)

Some examples (on the integers): \"$=$\" is an equivalence relation,
\"$<$\" does not satisfy symmetry, \"same sign\" is a equivalence, while
\"nearer than $10$\" fails transitivity.

### Partitions

In \"same sign\" $\{ (1,3),(-5,-7),(-1,-1),\ldots\}$ there are two kinds
of pairs, the first with both numbers positive and the second with both
negative. So integers fall into exactly one of two classes, positive or
negative.

A **partition** of a set $S$ is a collection of subsets
$\{S_1,S_2,\ldots\}$ such that every element of $S$ is in one and only
one $S_i$: $S_1\cup S_2\cup \ldots{} = S$, and if $i$ is not equal to
$j$ then $S_i\cap S_j=\varnothing$. Picture $S$ being decomposed into
distinct parts.

```{=html}
<center>
```
![](Linalg_partition.png "Linalg_partition.png"){width="" height="200"}

```{=html}
</center>
```
Thus, the first paragraph says \"same sign\" partitions the integers
into the positives and the negatives.

Similarly, the equivalence relation \"=\" partitions the integers into
one-element sets.

Another example is the fractions. Of course, $2/3$ and $4/6$ are
equivalent fractions. That is, for the set
$S=\{n/d\,\big|\, n,d\in\mathbb{Z}\text{ and }d\neq 0\}$, we define two
elements $n_1/d_1$ and $n_2/d_2$ to be equivalent if $n_1d_2=n_2d_1$. We
can check that this is an equivalence relation, that is, that it
satisfies the above three conditions. With that, $S$ is divided up into
parts.

```{=html}
<center>
```
![](Linalg_partition_2.png "Linalg_partition_2.png"){width=""
height="200"}

```{=html}
</center>
```
Before we show that equivalence relations always give rise to
partitions, we first illustrate the argument. Consider the relationship
between two integers of \"same parity\", the set
$\{ (-1,3),(2,4),(0,0),\ldots\}$ (i.e., \"give the same remainder when
divided by $2$\"). We want to say that the natural numbers split into
two pieces, the evens and the odds, and inside a piece each member has
the same parity as each other. So for each $x$ we define the set of
numbers associated with it:
$S_x=\{y\,\big|\, (x,y)\in\text{same parity}\}$. Some examples are
$S_1=\{\ldots,-3,-1,1,3,\ldots\}$, and $S_4=\{\ldots,-2,0,2,4,\ldots\}$,
and $S_{-1}=\{\ldots,-3,-1,1,3,\ldots\}$. These are the parts, e.g.,
$S_1$ is the odds.

}}`<b>`{=html}Theorem`</b>`{=html} An equivalence relation induces a
partition on the underlying set.

}}We call each part of a partition an **equivalence class** (or
informally, \"part\").

We somtimes pick a single element of each equivalence class to be the
**class representative**.

```{=html}
<center>
```
![](Linalg_partition_3.png "Linalg_partition_3.png"){width=""
height="200"}

```{=html}
</center>
```
Usually when we pick representatives we have some natural scheme in
mind. In that case we call them the
**canonical** representatives.

An example is the simplest form of a fraction. We\'ve defined $3/5$ and
$9/15$ to be equivalent fractions. In everyday work we often use the
\"simplest form\" or \"reduced form\" fraction as the class
representatives.

```{=html}
<center>
```
![](Linalg_partition_4.png "Linalg_partition_4.png"){width=""
height="200"}

```{=html}
</center>
```
|current=Sets, Functions, Relations
|previous=Techniques of Proof
|next=Licensing And History}}`{=mediawiki}


# Circuit Theory/All Chapters

\

 This wikibook is going
to be an introductory text about electric circuits. It will cover some
the basics of electric circuit theory, circuit analysis, and will touch
on circuit design. This book will serve as a companion reference for a
1st year of an Electrical Engineering undergraduate curriculum. Topics
covered include AC and DC circuits, passive circuit components, phasors,
and RLC circuits. The focus is on students of an electrical engineering
undergraduate program. Hobbyists would benefit more from reading
Electronics instead.

*This book is not nearly completed, and could still be improved. People
with knowledge of the subject are encouraged to contribute.*

The main editable text of this book is located at
<http://en.wikibooks.org/wiki/Circuit_Theory>. The wikibooks version of
this text is considered the most up-to-date version, and is the best
place to edit this book and contribute to it.


# Circuit Theory/Circuit Theory Introduction

## Who is This Book For?

This is designed for a first course in Circuit Analysis which is usually
accompanied by a set of labs. It is assumed that students are in a
Differential Equations class at the same time. Phasors are used to avoid
the Laplace transform of driving functions while maintaining a complex
impedance transform of the physical circuit that is identical in both.
1st and 2nd order differential equations can be solved using phasors and
calculus if the driving functions are sinusoidal. The sinusoidal is then
replaced by the more simple step function and then the convolution
integral is used to find an analytical solution to any driving function.
This leaves time for a more intuitive understanding of poles, zeros,
transfer functions, and Bode plot interpretation.

For those who have already had differential equations, the Laplace
transform equivalent will be presented as an alternative while focusing
on phasors and calculus.

This book will expect the reader to have a firm understanding of
Calculus specifically, and will not stop to explain the fundamental
topics in Calculus.

For information on Calculus, see the wikibook:
Calculus.

## What Will This Book Cover?

This book will cover linear circuits, and linear circuit elements.

The goal is to emphasize Kirchhoff and symbolic algebra
systems such as
MuPAD,
Mathematica or
Sage; at the expense of analysis methods such
as node,
mesh, and Norton
equivalent. A phasor/calculus based
approach starts at the very beginning and ends with the convolution
integral to handle all the various types of forcing functions.

The result is a linear analysis experience that is general in nature but
skips Laplace and Fourier transforms.

Kirchhoff\'s laws receive normal focus, but the other circuit
analysis/simplification techniques receive less than a normal attention.

The class ends with application of these concepts in Power Analysis,
Filters, Control systems.

The goal is set the ground work for a transition to the digital version
of these concepts from a firm basis in the physical world. The next
course would be one focused on modeling linear systems and analyzing
them digitally in preparation for a digital signal
(DSP) processing course.

## Where to Go From Here

-   For a technician version of this course which focuses on the
    practical rather than the idea, expertise rather than theory, and on
    algebra rather than calculus, see the
    **Electronics** wikibook.
-   Take **Ordinary Differential
    Equations** now that
    you have a practical reason to do so. This math class should ideally
    feel like an \"art\" class and be enjoyable.
-   To begin a course of study in Computer Engineering, see the
    **Digital Circuits** wikibook.
-   For a traditional \"next\" course, see the **Signals and
    Systems** wikibook. There should be
    an overlap with this wikibook.
-   The intended next course would have a name such as **A New Sequence
    in Signals and Linear
    Systems**
    or **Elements of Discrete Signal
    Analysis**.


# Circuit Theory/Terminology

## Basic Terminology

There are a few key terms that need to be understood at the beginning of
this book, before we can continue. This is only a partial list of all
terms that will be used throughout this book, but these key words are
important to know before we begin the main narrative of this text.

Time domain : The **time domain** is described by graphs of power, voltage and current that depend upon time. This is simply another way of saying that our circuits change with time, and that the major variable used to describe the system is time. Another name is \"temporal\".

```{=html}
<!-- -->
```

Frequency domain: The **frequency domain** are graphs of power, voltage and/or current that depend upon frequency such as Bode plots. Variable frequencies in wireless communication can represent changing channels or data on a channel. Another name is the \"Fourier domain\". Other domains that an engineer might encounter are the \"Laplace domain\" (or the \"s domain\" or \"complex frequency domain\"), and the \"Z domain\". When combined with the time, it is called a \"spectral\" or \"waterfall.\"

```{=html}
<!-- -->
```

Circuit response : Circuits generally have inputs and outputs. In fact, it is safe to say that a circuit isn\'t useful if it doesn\'t have one or the other (usually both). **Circuit response** is the relationship between the circuit\'s input to the circuit\'s output. The circuit response may be a measure of either current or voltage.

```{=html}
<!-- -->
```

Non-homogeneous : Circuits are described by equations that capture the component characteristics and how they are wired together. These equations are **non-homogeneous** in nature. Solving these equations requires splitting the single problem into two problems: a *steady-state solution* (particular solution) and a *transient solution* (homogeneous solution).

```{=html}
<!-- -->
```

Steady-state solution : The final value, when all circuit elements have a constant or periodic behavior, is also known as the steady-state value of the circuit. The circuit response at steady-state (when voltages and currents have stopped changing due to a disturbance) is also known as the \"steady-state response\". The **steady-state solution** to the particular integral is called the *particular solution*.

```{=html}
<!-- -->
```

Transient : A **transient** event is a short-lived burst of energy in a system caused by a sudden change of state.

```{=html}
<!-- -->
```

:   Transient "wikilink") means momentary,
    or a short period of time. Transient means that the energy in a
    circuit suddenly changes which causes the energy storage elements to
    react. The circuit\'s energy state is forced to change. When a car
    goes over a bump, it can fly apart, feel like a rock, or cushion the
    impact in a designed manner. The goal of most circuit design is to
    plan for transients, whether intended or not.

```{=html}
<!-- -->
```

:   Transient solutions are determined by assuming the driving
    function(s) is zero which creates a homogeneous equation, which has
    a **homogeneous
    solution**.

## Summary

When something changes in a circuit, there is a certain transition
period before a circuit \"settles down\", and reaches its final value.
The response that a circuit has before settling into its *steady-state
response* is known as the *transient response*. Using **Euler\'s
formula**, **complex
numbers**,
**phasors** and the
**s-plane**, a **homogeneous
solution**
technique will be developed that captures the transient response by
assuming the final state has no energy. In addition, a **particular
solution**
technique will be developed that finds the final energy state. Added
together, they predict the *circuit response*.

The related **Differential
equation** development of
homogeneous and particular solutions will be avoided.