Ordered ring

An **ordered ring** is a [https://arbital.com/p/-3gq](https://arbital.com/p/-3gq) $R=(X,\oplus,\otimes)$ with a [total order](https://arbital.com/p/540) $\leq$ compatible with the ring structure. Specifically, it must satisfy these axioms for any $a,b,c \in X$:

- If $a \leq b$, then $a \oplus c \leq b \oplus c$.
- If $0 \leq a$ and $0 \leq b$, then $0 \leq a \otimes b$

An element $a$ of the ring is called "[https://arbital.com/p/-positive](https://arbital.com/p/-positive)" if $0<a$ and "[https://arbital.com/p/-negative](https://arbital.com/p/-negative)" if $a<0$. The second axiom, then, says that the product of nonnegative elements is nonnegative.

An ordered ring that is also a [field](https://arbital.com/p/481) is an [https://arbital.com/p/-ordered_field](https://arbital.com/p/-ordered_field).

#Basic Properties

- For any element $a$, $a \leq 0$ if and only if $0 \leq -a$.

%%hidden(Show proof):
First suppose $a \leq 0$. Using the first axiom to add $-a$ to both sides, $a+(-a) = 0 \leq -a$. For the other direction, suppose $0 \leq -a$. Then $a \leq -a+a = 0$.
%%

- The product of nonpositive elements is nonnegative.

%%hidden(Show proof):
Suppose $a$ and $b$ are nonpositive elements of $R$, that is $a,b \leq 0$. From the first axiom, $a+(-a) = 0 \leq -a$, and similarly $0 \leq -b$. By the second axiom $0 \leq -a \otimes -b$. But $-a \otimes -b = a \otimes b$, so $0 \leq a \otimes b$.
%%

- The [https://arbital.com/p/-square](https://arbital.com/p/-square) of any element is nonnegative.

%%hidden(Show proof):
Let $a$ be such an element. Since the ordering is total, either $0 \leq a$ or $a \leq 0$. In the first case, the second axiom gives $0 \leq a^2$. In the second case, the previous property gives $0 \leq a^2$, since $a$ is nonpositive. Either way we have $0 \leq a^2$.
%%

- The additive [identity](https://arbital.com/p/54p) $1 \geq 0$. (Unless the ring is trivial, $1>0$.)

%%hidden(Show proof):
Clearly $1 = 1 \otimes 1$. So $1$ is a square, which means it's nonnegative.
%%

# Examples

The [real numbers](https://arbital.com/p/4bc) are an ordered ring (in fact, an [https://arbital.com/p/-ordered_field](https://arbital.com/p/-ordered_field)), as is any [https://arbital.com/p/-subring](https://arbital.com/p/-subring) of $\mathbb R$, such as [$\mathbb Q$](https://arbital.com/p/4zq).

The [complex numbers](https://arbital.com/p/4zw) are not an ordered ring, because there is no way to define the order between $0$ and $i$. Suppose that $0 \le i$, then, we have $0 \le i \times i = -1$, which is false. Suppose that $i \le 0$, then $0 = i + (-i) \le 0 + (-i)$, but then we have $0 \le (-i) \times (-i) = -1$, which is again false. Alternatively, $i^2=-1$ is a square, so it must be nonnegative; that is, $0 \leq -1$, which is a contradiction.

An **ordered ring** is a [https://arbital.com/p/-3gq](https://arbital.com/p/-3gq) that is [totally ordered](https://arbital.com/p/540), where the ordering agrees with the ring operations. In particular, adding something to two elements doesn't change which of them is bigger, and the product of two [https://arbital.com/p/-positive](https://arbital.com/p/-positive) elements is positive.


Addition of rational numbers (Math 0)

We've already met the idea that rational numbers are what we can make by putting together the building blocks which we get by dividing a single apple into some number of equally-sized pieces; and remember that the building blocks have a special notation, as $\frac{1}{\text{number}}$.
Recall also that if we take $5$ of a certain building block, for instance, we write $\frac{5}{\text{number}}$.

It's clear that if you take one apple and another apple, and put them together, you'll get two apples.
So we should hope that that's the same if we divided up the apples into pieces first (without removing any of the pieces).

We write $a+b$ for "take the rational number $a$ and put it next to the rational number $b$, and count up what you've got".
Because I'm bad at drawing, we'll pretend apples are just perfect circles. %%note:This is actually a design decision: eventually we'll want to get away from considering apples, and this more abstract representation will be useful.%%

![Example of a sum of two rational numbers](http://i.imgur.com/GUgmCwb.png)

----------

For example, we should hope that $\frac{2}{2} + \frac{3}{3} = 2$.

![Two halves and three thirds](http://i.imgur.com/6vdRdcT.png)

What about in cases which aren't of the form $\frac{n}{n}$ for some integer $n$?
Well, how about $\frac{5}{3} + \frac{8}{3}$.

![Five thirds and eight thirds](http://i.imgur.com/7dPqmCP.png)

If we had five $\frac{1}{3}$-blocks, and we put them together with another eight $\frac{1}{3}$-blocks, we would hope to have $5+8=13$ blocks.
So we should hope that $\frac{5}{3} + \frac{8}{3} = \frac{13}{3}$.

![Thirds, separated out](http://i.imgur.com/foVkEiS.png)

And in general, if both our quantities are made up from the same size of building-block (in the above case, the blocks are $\frac{1}{3}$-sized), we should just be able to take the two numerators %%note:Remember, that was the word mathematicians use for the number of blocks we have.%% and add them together.

----------

Very well. But what if the two quantities are not made up from the same size of building-block? That is, the denominators are different?
For example, $\frac{5}{3} + \frac{5}{4}$?

![Five thirds and five quarters](http://i.imgur.com/xFtKEnU.png)

![Five thirds and five quarters, rearranged](http://i.imgur.com/Av984T0.png)

Now it's not so clear.
How might we approach this? You should muse on this for thirty seconds before reading on; it will be good for your soul.

%%hidden(Show solution):
The way we do it is to find some *smaller* block-size, out of which we can make *both* building-blocks.
%%

# Example

Our example is $\frac{5}{3} + \frac{5}{4}$.
So we want to make the $\frac{1}{3}$ block and the $\frac{1}{4}$ block out of some smaller block.

Now, I won't tell you yet how I got this, but you can check for yourself that $\frac{1}{12}$-blocks will make both $\frac{1}{3}$- and $\frac{1}{4}$-blocks: because three $\frac{1}{12}$-blocks together make something the same size as a $\frac{1}{4}$-block, and four $\frac{1}{12}$-blocks together make something the same size as a $\frac{1}{3}$-block.

![Thirds, divided into twelfths](http://i.imgur.com/k2iz8uR.png)

![Quarters, divided into twelfths](http://i.imgur.com/gmHK3oj.png)

Therefore $\frac{1}{3} = \frac{4}{12}$:

![One third, divided into twelfths](http://i.imgur.com/fFX4Dmc.png)

And $\frac{1}{4} = \frac{3}{12}$:

![One quarter, divided into twelfths](http://i.imgur.com/gRqmfAc.png)

Now, if $\frac{1}{3} = \frac{4}{12}$ - that is, if one $\frac{1}{3}$-piece is the same size as four $\frac{1}{12}$-pieces - then it should be the case that five $\frac{1}{3}$-pieces come to five lots of four $\frac{1}{12}$-pieces.
That is, to twenty $\frac{1}{12}$-pieces: $\frac{5}{3} = \frac{20}{12}$.

Similarly, $\frac{5}{4} = \frac{15}{12}$, because five $\frac{1}{4}$-pieces is the same as five lots of three sets of $\frac{1}{12}$-pieces, and $5 \times 3 = 15$.

Therefore $\frac{5}{3} + \frac{5}{4}$ should be just the same as $\frac{20}{12} + \frac{15}{12}$, which we know how to calculate!
It is $\frac{35}{12}$.

# General procedure

The preceding example might suggest a general way for adding two fractions together. (In the process, this should put to rest the existential dread mentioned in [the intro to rational numbers](https://arbital.com/p/4zx), about whether it makes sense to add two rational numbers at all.)

The general procedure is as follows:

 1. Find a building-block size such that the building-blocks of both fractions can be made out of that building-block
 2. Express each fraction individually in terms of that building-block
 3. Add the two together (which we can now do, because they are both made of the same size of building-block).

The interesting part is really the first step.

## Finding a building-block that will neatly make two other differently-sized blocks

Let's be concrete here: just as above, we showed that $\frac{1}{12}$ works for blocks of size $\frac{1}{3}$ and $\frac{1}{4}$, so we'll now consider $\frac{1}{2}$ and $\frac{1}{5}$.

We're trying to divide up $\frac{1}{2}$ into copies of a smaller building-block, such that with copies of that *same* building block, we can also make $\frac{1}{5}$.
Here is the key insight that tells us how to do it: let us pretend that we are dividing up the *same* object, this time a square instead of a circle, into two pieces along one edge and into five pieces along the other edge.
How many pieces have we made?

![Square divided into tenths](http://i.imgur.com/qkMkQvZ.png)

There are $2 \times 5 = 10$ pieces, each of them the same size (namely the $\frac{1}{10}$-block size), and note that as if by magic we've just produced a way to subdivide $\frac{1}{2}$ into pieces made up of the $\frac{1}{10}$-block!

![Square divided into tenths, one half indicated](http://i.imgur.com/n9dYZBA.png)

And we also have the $\frac{1}{5}$ subdivided in the same way!

![Square divided into tenths, one fifth indicated](http://i.imgur.com/0ny9RnS.png)

This might suggest a general way to do this; you should certainly ponder for thirty seconds before continuing.
%%hidden(Show the general way):

Let's say we are trying to make both $\frac{1}{m}$ and $\frac{1}{n}$-sized blocks out of smaller blocks.
(So above, $m$ is standing for $2$ and $n$ is standing for $5$.)

Then we can do this by splitting both up into $\frac{1}{m \times n}$-blocks.

Indeed, $\frac{1}{n} = \frac{m}{m \times n}$ (i.e. we can make a $\frac{1}{n}$-block out of $m$ of the $\frac{1}{m \times n}$ tiny-blocks),
and $\frac{1}{m} = \frac{n}{m \times n}$ (i.e. we can make a $\frac{1}{m}$-block out of $n$ of the $\frac{1}{m \times n}$ tiny-blocks).
%%

## Aside: arithmetical rules

The description of the "general way" above has become pretty full of $\frac{1}{\text{thing}}$, and refers less and less to the building-blocks we originally considered.
The reason for this is that $\frac{1}{\text{thing}}$ is an extremely convenient shorthand, and the "general way" can be written as a very simple rule for manipulating that shorthand.
Once we have such a convenient rule, we can compute with fractions without having to go through the bother of working out what each fraction really and truly *means* (in terms of the building-blocks); this saves a lot of time in practice.
As an analogy, it's easier for you to drive a car if, every time you change gears, you *don't* have to work out what's happening inside the gearbox.

The simple rule we have just seen is $$\frac{1}{m} + \frac{1}{n} = \frac{n}{m \times n} + \frac{m}{m \times n}$$
And we've already had the rule that $$\frac{a}{m} + \frac{b}{m} = \frac{a+b}{m}$$
(Remember, if these are not obvious to you, you should think back to what they really mean, in terms of the building-blocks. The first rule is to do with dividing up a big square into little squares, from the previous subsection; the second rule is just taking two collections of building-blocks of the same size and putting them together.)

This is almost every rule we need to add any pair of fractions!
We can now add single building blocks of any size together.

To add any pair of rational numbers together - say of building-block size $\frac{1}{m}$ and $\frac{1}{n}$ respectively - we just need to express them each in terms of the smaller building-block $\frac{1}{m \times n}$, and add them together as normal.

%%hidden(Example):
Earlier, we considered $\frac{5}{4} + \frac{5}{3}$.

A building-block which can make both the $\frac{1}{4}$ and $\frac{1}{3}$-sized blocks is the $\frac{1}{12}$-block, because $3 \times 4 = 12$.

Therefore we need to express both our fractions as being made from $\frac{1}{12}$-sized blocks.

$\frac{5}{4}$ is $\frac{15}{12}$, because in each of the five $\frac{1}{4}$-blocks, there are three $\frac{1}{12}$-blocks, so we have a total of $5 \times 3$ blocks of size $\frac{1}{12}$.

$\frac{5}{3}$ is $\frac{20}{12}$, because in each of the five $\frac{1}{3}$-blocks, there are four $\frac{1}{12}$-blocks, so we have a total of $5 \times 4 = 20$ blocks of size $\frac{1}{12}$.

Therefore our final answer is $\frac{15}{12} + \frac{20}{12} = \frac{35}{12}$.
%%

To make this into a general rule: $$\frac{a}{m} + \frac{b}{n} = \frac{a \times n}{m \times n} + \frac{b \times m}{m \times n} = \frac{a \times n + b \times m}{m \times n}$$

(Recall the order of operations in the integers: the notation $a \times n + b \times m$ means "do $a \times n$; then do $b \times m$; then finally add them together". Multiplication comes before addition.)

And that's it! That's how we add rational numbers together, and it works even when any or all of $a, b, m, n$ are negative.
(Remember, though, that $m$ and $n$ can't be zero, because it makes no sense to divide something up into no pieces.)

It would be wise now to try the [exercises](https://arbital.com/p/55p). One learns mathematics through doing, much more than through simply reading; your understanding will be cemented by going through some concrete examples.

Addition is "putting two quantities next to each other and working out how much they come to when combined".


Addition of rational numbers exercises

This page consists of exercises designed to help you get to grips with the addition of rational numbers.

# Core exercises

## First exercise: $\frac{1}{10} + \frac{1}{5}$

%%hidden(Show some possible solutions):
Using the instant rule from the text (which is actually a bit unwieldy here): $$\frac{1}{10} + \frac{1}{5} = \frac{1 \times 5 + 10 \times 1}{10 \times 5} = \frac{5+10}{50} = \frac{15}{50}$$
Notice that this can actually be made simpler: it is the same thing as $\frac{3}{10}$, because when we take $\frac{3}{10}$ and split each $\frac{1}{10}$-block into five equal pieces, we obtain $15$ copies of the $\frac{1}{50}$-block.

Alternatively, one could spot that $\frac{1}{10}$-blocks actually can already be used to make $\frac{1}{5}$-blocks: $\frac{1}{5} = \frac{2}{10}$.
Therefore we're actually trying to find $\frac{1}{10} + \frac{2}{10}$, which is easy: it's $\frac{3}{10}$.
%%

## Second exercise: $\frac{1}{15} + \frac{1}{10}$

%%hidden(Show some possible solutions):
Using the instant rule from the text: $$\frac{1}{10} + \frac{1}{15} = \frac{1 \times 15 + 10 \times 1}{10 \times 15} = \frac{25}{150} = \frac{1}{6}$$

Were you expecting a big denominator, or at least a multiple of 5? From this example, we can see that the final answer of a rational addition problem can have a denominator which doesn't even seem related to the others.

Alternatively, one could spot that $\frac{1}{30}$-blocks will make up $\frac{1}{10}$- and $\frac{1}{15}$-blocks.
Then we are actually trying to find $\frac{3}{30} + \frac{2}{30} = \frac{5}{30}$; it's a bit easier to see that $\frac{5}{30} = \frac{1}{6}$ than it is to see that $\frac{25}{150} = \frac{1}{6}$.
%%

## Third exercise: $\frac{1}{10} + \frac{1}{15}$
You might find this exercise a little familiar…

%%hidden(Show possible solution):
Using the instant rule from the text: $$\frac{1}{15} + \frac{1}{10} = \frac{1 \times 10 + 15 \times 1}{15 \times 10} = \frac{25}{150} = \frac{1}{6}$$

You may notice that we've basically done the same calculations as in the second exercise.
In fact, addition doesn't care which way round the numbers go: $\frac{1}{10} + \frac{1}{15} = \frac{1}{15} + \frac{1}{10}$, even if we don't already know that that number is $\frac{1}{6}$.

This is intuitive from the fact that addition is the idea of "place the apples next to each other and count up the total": just putting the apples down in a different order doesn't change the total amount of apple.
%%

## Fourth exercise: $\frac{0}{5} + \frac{2}{5}$

%%hidden(Show possible solution):
Notice that both the denominators are the same (namely $5$), so we can just combine the $\frac{1}{5}$-sized pieces straight away.
We have $0$ pieces and $2$ pieces, so the total is $2$ pieces.

That is, the answer is $\frac{2}{5}$.
%%

## Fifth exercise: $\frac{0}{7} + \frac{2}{5}$

%%hidden(Show possible solution):
If you spot that this is "no $\frac{1}{7}$-pieces" next to "two $\frac{1}{5}$-pieces", then you might just immediately write down that the answer is $\frac{2}{5}$ because there aren't any $\frac{1}{7}$-pieces to change the answer; and you'd be correct.

To use the instant rule from the text: $$\frac{0}{7} + \frac{2}{5} = \frac{0 \times 5 + 2 \times 7}{5 \times 7} = \frac{0 + 14}{35} = \frac{14}{35}$$

But that is the same as $\frac{2}{5}$ (simply expressed with each $\frac{1}{5}$-piece subdivided further into sevenths).
%%

# Extension exercises

These exercises are meant to be harder and to stretch your conceptual understanding.
Give them a proper go, but don't worry too much if you don't get the same answers as me.
Mine are, in a technical sense, "right", but no matter what you end up with, you will derive a lot of benefit from trying to work out what the answers are yourself without having been told exactly how.
The learning of mathematics is much more about thinking and understanding (usually guided by examples) than it is about just repeatedly carrying out calculations.

## First extension exercise: $\frac{1}{5} + \frac{-1}{10}$
Yes, it is possible to add a negative number of chunks.
Try using the instant rule and see what happens.

%%hidden(Show possible solution):
Using the instant rule from the text: $$\frac{1}{15} + \frac{-1}{10} = \frac{1 \times 10 + 15 \times (-1)}{15 \times 10} = \frac{10 - 15}{150} = \frac{-5}{150} = \frac{-1}{30}$$

What has happened here? What have we "really done" with our chunks of apple?
Have a think; we'll see a lot more of this when we get to [subtraction](https://arbital.com/p/56x).
%%

## Second extension exercise: what rational number must we add to $\frac{7}{8}$ to obtain $\frac{13}{8}$?

You're looking for an answer that looks like $\frac{a}{b}$ where there are integers in place of $a$ and $b$.

%%hidden(Show some possible solutions):

Something you can do to make this question easier is to notice that both the numbers have the same chunk-size (namely $\frac{1}{8}$), so we might try adding some number of $\frac{1}{8}$-chunks.
Then we're trying to get from $7$ chunks to $13$ chunks, so we need to add $6$ chunks.

That is, the final number is $\frac{6}{8}$ (which is also $\frac{3}{4}$).
%%

## Third extension exercise: what rational number must we add to $\frac{7}{8}$ to obtain $\frac{13}{7}$?

You're looking for an answer that looks like $\frac{a}{b}$ where there are integers in place of $a$ and $b$.

%%hidden(Show some possible solutions):

Now, the numbers are no longer of the same chunk-size, so we should make it so that they are of the same size.

The chunk-size to use is $\frac{1}{8 \times 7} = \frac{1}{56}$. The reason for this is the same as the reasoning we saw when working out how to add $\frac{1}{8}$ and $\frac{1}{7}$.

Then the two numbers are $\frac{7 \times 7}{7 \times 8} = \frac{49}{56}$ and $\frac{8 \times 13}{8 \times 7} = \frac{104}{56}$, so the answer is that we need to get from $49$ to $104$; to do that, we need to add $55$ chunks of size $\frac{1}{56}$, so the answer is $\frac{55}{56}$.
%%


Well-ordered set

A **well-ordered set** is a [https://arbital.com/p/-540](https://arbital.com/p/-540) $(S, \leq)$, such that for any nonempty subset $U \subset S$ there is some $x \in U$ such that for every $y \in U$, $x \leq y$; that is, every nonempty subset of $S$ has a least element.

Any finite totally ordered set is well-ordered. The simplest [infinite](https://arbital.com/p/infinity) well-ordered set is [$\mathbb N$](https://arbital.com/p/45h), also called [$\omega$](https://arbital.com/p/ordinal_omega) in this context.

Every well-ordered set is [isomorphic](https://arbital.com/p/4f4) to a unique [https://arbital.com/p/-ordinal_number](https://arbital.com/p/-ordinal_number), and thus any two well-ordered sets are comparable.

The order $\leq$ is called a "well-ordering," despite the fact that "well" is usually an adverb.

#Induction on a well-ordered set

[https://arbital.com/p/mathematical_induction](https://arbital.com/p/mathematical_induction) works on any well-ordered set. On well-ordered sets longer than $\mathbb N$, this is called [https://arbital.com/p/-transfinite_induction](https://arbital.com/p/-transfinite_induction).

Induction is a method of proving a statement $P(x)$ for all elements $x$ of a well-ordered set $S$. Instead of directly proving $P(x)$, you prove that if $P(y)$ holds for all $y < x$, then $P(x)$ is true. This suffices to prove $P(x)$ for all $x \in S$.

%%hidden(Show proof):
Let $U = \{x \in S \mid \neg P(x) \}$ be the set of elements of $S$ for which $P$ doesn't hold, and suppose $U$ is nonempty. Since $S$ is well-ordered, $U$ has a least element $x$. That means $P(y)$ is true for all $y < x$, which implies $P(x)$. So $x \not\in U$, which is a contradiction. Hence $U$ is empty, and $P$ holds on all of $S$.
%%


Needs parent

This page is not attached to an appropriate parent page.

Dylan Hendrickson
Eric Bruylant
Patrick Stevens


Löb's theorem

We trust Peano Arithmetic to correctly capture certain features of the [standard model of arithmetic](https://arbital.com/p/). Furthermore, we know that Peano Arithmetic is expressive enough to [talk about itself](https://arbital.com/p/31z) in meaningful ways. So it would certainly be great if Peano Arithmetic asserted what now is an intuition: that everything it proves is certainly true.

In formal notation, let $Prv$ stand for the [https://arbital.com/p/-5gt](https://arbital.com/p/-5gt) of $PA$. Then, $Prv(T)$ is true if and only if there is a proof from the axioms and rules of inference of $PA$ of $T$. Then what we would like $PA$ to say is that $Prv(S)\implies S$ for every sentence $S$.

But alas, $PA$ suffers from a problem of self-trust.

Löb's theorem states that if $PA\vdash Prv(S)\implies S$ then $PA\vdash S$. This immediately implies that if $PA$ is consistent, the sentences $PA\vdash Prv(S)\implies S$ are not provable when $S$ is false, even though according to our intuitive understanding of the standard model every sentence of this form must be true.

Thus, $PA$ is incomplete, and fails to prove a particular set of sentences that would increase massively our confidence in it.

Notice that [Gödel's second incompleteness theorem](https://arbital.com/p/godels_second_incompleteness_theorem) follows immediately from Löb's theorem, as if $PA$ is consistent, then by Löb's $PA\nvdash Prv(0= 1)\implies 0= 1$, which by the propositional calculus implies $PA\nvdash \neg Prv(0= 1)$.

It is worth remarking that Löb's theorem does not only apply to the standard provability predicate, but to every predicate satisfying the [Hilbert-Bernais derivability conditions](https://arbital.com/p/).

Patrick LaVictoire
Eric Rogstad
Malcolm McCrimmon
Eric Bruylant
Jaime Sevilla Molina

If $PA\vdash Prv_{PA}(A)\implies A$ then $PA\vdash A$


Frequency diagrams: A first look at Bayes

Bayesian reasoning is about how to revise our beliefs in the light of evidence.

We'll start by considering one scenario in which the strength of the evidence has clear numbers attached.

(Don't worry if you don't know how to solve the following problem.  We'll see shortly how to solve it.)

Suppose you are a nurse screening a set of students for a sickness called Diseasitis.%note: Literally "inflammation of the disease".%

- You know, from past population studies, that around 20% of the students will have Diseasitis at this time of year.

You are testing for Diseasitis using a color-changing tongue depressor, which usually turns black if the student has Diseasitis.

- Among patients with Diseasitis, 90% turn the tongue depressor black.
- However, the tongue depressor is not perfect, and also turns black 30% of the time for healthy students.

One of your students comes into the office, takes the test, and turns the tongue depressor black.  What is the probability that they have Diseasitis?

(*If* you think you see how to do it, you can try to solve this problem before continuing.  To quickly see if you got your answer right, you can expand the "Answer" button below; the derivation will be given shortly.)

%hidden(Answer): The probability a student with a blackened tongue depressor has Diseasitis is 3/7, roughly 43%. %

This problem can be solved a hard way or a clever easy way.  We'll walk through the hard way first.

First, we imagine a population of 100 students, of whom 20 have Diseasitis and 80 don't.%%note:Multiple studies show that thinking about concrete numbers such as "20 out of 100 students" or "200 out of 1000 students" is more likely to produce correct spontaneous reasoning on these problems than thinking about percentages like "20% of students." E.g. "[Probabilistic reasoning in clinical medicine](https://faculty.washington.edu/jmiyamot/p548/eddydm%20prob%20reas%20i%20clin%20medicine.pdf)" by David M. Eddy (1982).%%

![prior frequency](https://i.imgur.com/gvcoqQN.png)

90% of sick students turn their tongue depressor black, and 30% of healthy students turn the tongue depressor black.  So we see black tongue depressors on 90% * 20 = 18 sick students, and 30% * 80 = 24 healthy students.

![posterior frequency](https://i.imgur.com/0XDbrYi.png)

What's the probability that a student with a black tongue depressor has Diseasitis? From the diagram, there are 18 sick students with black tongue depressors.  18 + 24 = 42 students in total turned their tongue depressors black.  Imagine reaching into a bag of all the students with black tongue depressors, and pulling out one of those students at random; what's the chance a student like that is sick?

![conditional probability](http://i.imgur.com/W6avfHO.png?1)

The final answer is that a patient with a black tongue depressor has an 18/42 = 3/7 = 43% probability of being sick.

Many medical students have at first found this answer counter-intuitive:  The test correctly detects Diseasitis 90% of the time! If the test comes back positive, why is it still less than 50% likely that the patient has Diseasitis? Well, the test also incorrectly "detects" Diseasitis 30% of the time in a healthy patient, and we start out with lots more healthy patients than sick patients.

The test does provide *some* evidence in favor of of the patient being sick.  The probability of a patient being sick goes from 20% before the test, to 43% after we see the tongue depressor turn black.  But this isn't conclusive, and we need to perform further tests, maybe more expensive ones.

*If* you feel like you understand this problem setup, consider trying to answer the following question before proceeding: What's the probability that a student who does *not* turn the tongue depressor black - a student with a negative test result - has Diseasitis?  Again, we start out with 20% sick and 80% healthy students, 70% of healthy students will get a negative test result, and only 10% of sick students will get a negative test result.

%%hidden(Answer):Imagine 20 sick students and 80 healthy students.  10% * 20 = 2 sick students have negative test results.  70% * 80 = 56 healthy students have negative test results.  Among the 2+56=58 total students with negative test results, 2 students are sick students with negative test results.  So 2/58 = 1/29 = 3.4% of students with negative test results have Diseasitis.%%

%if-before([https://arbital.com/p/1x1](https://arbital.com/p/1x1)): Now let's turn to a faster, easier way to solve the same problem.%
%!if-before([https://arbital.com/p/1x1](https://arbital.com/p/1x1)): For a more clever way to perform the same calculation, see [https://arbital.com/p/1x1](https://arbital.com/p/1x1).%

Alexei Andreev
Eric Rogstad
Sachin Krishnan
Anthony Mercuri
Akanksha rawat
Nate Soares
jj jj
David Salamon
Eric Bruylant
John Mahony
Eliezer Yudkowsky


Frequency diagram

Frequency diagrams, like [waterfall diagrams](https://arbital.com/p/1wy), provide a way of visualizing [Bayes' Rule](https://arbital.com/p/1lz).  For example, if 20% of the patients in the screening patient are sick (red) and 80% are healthy (blue); and 90% of the sick patients get positive test results; and 30% of the healthy patients get positive test results, we could visualize the probabilities as proportions of a large population, as in the following frequency diagram:

![Frequency diagram](https://i.imgur.com/0XDbrYi.png)

See [https://arbital.com/p/55z](https://arbital.com/p/55z) for a walkthrough of the diagram.

Eric Bruylant
Nate Soares
Alexei Andreev

Frequency diagrams, like [waterfall diagrams](https://arbital.com/p/1wy), provide a way of visualizing [Bayes' Rule](https://arbital.com/p/1lz).  For example, if 20% of the patients in the screening patient are sick (red) and 80% are healthy (blue); and 90% of the sick patients get positive test results; and 30% of the healthy patients get positive test results, we could visualize the probabilities as proportions of a large population, as in the following frequency diagram:

![Frequency diagram](https://i.imgur.com/0XDbrYi.png)

Then, when asking about the probability that a patient with a black tongue depressor is sick, we can divide the number of sick patients with black tongue depressors (18 in this case) by the number of total patients with black tongue depressors (18 + 24, in this case).


Odds: Introduction

Lets say we have a bag containing twice as many blue marbles as red marbles. Then, if you reach in without looking and pick out a marble at random, the odds are 2 : 1 in favor of drawing a blue marble as opposed to a red one.

Odds express *relative* quantities. 2 : 1 odds are the same as 4 : 2 odds are the same as 600 : 300 odds. For example, if the bag contains 1 red marble and 2 blue marbles, or 2 red marbles and 4 blue marbles, then your chance of pulling out a red marble is the same in both cases:

![](https://i.imgur.com/IcsOXl0.png?0)

In other words, given odds of $(x : y)$ we can scale it by any positive number $\alpha$ to get equivalent odds of $(\alpha x : \alpha y).$

# Converting odds to probabilities

If there were also green marbles, the *relative odds* for red *versus* blue would still be (1 : 2), but the *probability* of drawing a red marble would be lower.

![](https://i.imgur.com/ooyn9Py.png?0)

If red, blue, and green are the only kinds of marbles in the bag, then we can turn odds of $(r : b : g)$ into probabilities $(p_r : p_b : p_g)$ that say the probability of drawing each kind of marble.  Because red, blue, and green are the only possibilities, $p_r + p_g + p_b$ must equal 1, so $(p_r : p_b : p_g)$ must be odds equivalent to $(r : b : g)$ but "normalized" such that it sums to one. For example, $(1 : 2 : 1)$ would normalize to $\frac{1}{4} : \frac{2}{4} : \frac{1}{4},$ which are the probabilities of drawing a red / blue / green marble (respectively) from the bag on the right above.

Note that if red and blue are not the only possibilities, then it doesn't make sense to convert the odds $(r : b)$ of red vs blue into a probability. For example, if there are 100 green marbles, one red marble, and two blue marbles, then the odds of red vs blue are 1 : 2, but the probability of drawing a red marble is much lower than 1/3! Odds can only be converted into probabilities if its terms are [https://arbital.com/p/-1rd](https://arbital.com/p/-1rd).

Imagine a forest with some sick trees and some healthy trees, where the odds of a tree being sick (as opposed to heathy) are (2 : 3), and every tree is either sick or healthy (there are no in-between states). Then the probability of randomly picking a sick tree from among *all* trees is 2 / 5, because 2 out of every (2 + 3) trees is sick.

![](https://i.imgur.com/GVZnz2c.png?0)

In general, the operation we're doing here is taking relative odds like $(a : b : c \ldots)$ and dividing each term by the sum $(a + b + c \ldots)$ to produce $$\left(\frac{a}{a + b + c \ldots} : \frac{b}{a + b + c \ldots} : \frac{c}{a + b + c \ldots}\ldots\right)$$ Dividing each term by the sum of all terms gives us an equivalent set of odds (because each element is divided by the same amount) whose terms sum to 1.

This process of dividing a set of odds by the sum of its terms to get a set of probabilities that sum to 1 is called [normalization](https://arbital.com/p/1rk).

# Converting probabilities to odds

Let's say we have two events R and B, which might be things like "I draw a red marble" and "I draw a blue marble." Say $\mathbb P(R) = \frac{1}{4}$ and $\mathbb P(B) = \frac{1}{2}.$ What are the odds of R vs B? $\mathbb P(R) : \mathbb P(B) = \left(\frac{1}{4} : \frac{1}{2}\right),$ of course.

Equivalently, we can take the odds $\left(\frac{\mathbb P(R)}{\mathbb P(B)} : 1\right)$, because $\frac{\mathbb P(R)}{\mathbb P(B)}$ is how many more times likely R is than B. In this example, $\frac{\mathbb P(R)}{\mathbb P(B)} = \frac{1}{2},$ because R is half as likely as B. Sometimes, the quantity $\frac{\mathbb P(R)}{\mathbb P(B)}$ is called the "odds ratio of R vs B," in which case it is understood that the odds for R vs B are $\left(\frac{\mathbb P(R)}{\mathbb P(B)} : 1\right).$

# Odds to ratios

When there are only two terms $x$ and $y$ in a set of odds, the odds can be written as a ratio $\frac{x}{y}.$ The odds *ratio* $\frac{x}{y}$ refers to the *odds* $(x : y),$ or, equivalently, $\left(\frac{x}{y} : 1\right).$

Nate Soares
Eliezer Yudkowsky
Alexei Andreev


Odds: Refresher

Let's say that, in a certain forest, there are 2 sick trees for every 3 healthy trees. We can then say that the odds of a tree being sick (as opposed to healthy) are $(2 : 3).$

Odds express *relative* chances.  Saying "There's 2 sick trees for every 3 healthy trees" is the same as saying "There's 10 sick trees for every 15 healthy trees."  If the original odds are $(x : y)$ we can multiply by a positive number $\alpha$ and get a set of equivalent odds $(\alpha x : \alpha y).$

If there's 2 sick trees for every 3 healthy trees, and every tree is either sick or healthy, then the *probability* of randomly picking a sick tree from among *all* trees is 2/(2+3):

![Odds v probabilities](https://i.imgur.com/GVZnz2c.png?0)

If the set of possibilities $A, B, C$ are [mutually exclusive and exhaustive](https://arbital.com/p/1rd), then the probabilities $\mathbb P(A) + \mathbb P(B) + \mathbb P(C)$ should sum to $1.$  If there's no further possibilities $d,$ we can convert the relative odds $(a : b : c)$ into the probabilities $(\frac{a}{a + b + c} : \frac{b}{a + b + c} : \frac{c}{a + b + c}).$  The process of dividing each term by the sum of terms, to turn a set of proportional odds into probabilities that sum to 1, is called [normalization](https://arbital.com/p/1rk).

When there are only two terms $x$ and $y$ in the odds, they can be expressed as a single ratio $\frac{x}{y}.$  An odds ratio of $\frac{x}{y}$ refers to odds of $(x : y),$ or, equivalently, odds of $\left(\frac{x}{y} : 1\right).$ Odds of $(x : y)$ are sometimes called odds ratios, where it is understood that the actual ratio is $\frac{x}{y}.$


Conditional probability: Refresher

$\mathbb P(\text{left} \mid \text{right})$ is the probability that the thing on the left is true _assuming_ the thing on the right is true, and it's defined as $\frac{\mathbb P(\text{left} \land \text{right})}{\mathbb P(\text{right})}.$

Thus, $\mathbb P(yellow \mid banana)$ is the probability that a banana is yellow ("the probability of yellowness given banana"), while $\mathbb P(banana \mid yellow)$ is the probability that a yellow object is a banana ("the probability of banana, given yellowness").%%note: In general, $\mathbb P(v)$ is an abbreviation of $\mathbb P(V = v)$ for some variable $V$, which is assumed to be known from the context. For example, $\mathbb P(yellow)$ might stand for $\mathbb P({ColorOfNextObjectInBag}=yellow)$ where $ColorOfNextObjectInBag$ is a [variable](https://arbital.com/p/random_variable) in our [https://arbital.com/p/-probability_distribution](https://arbital.com/p/-probability_distribution) $\mathbb P,$ and $yellow$ is one of the [values](https://arbital.com/p/random_variable_value) that that variable can take on.%%

$\mathbb P(x \land y)$ is used to denote the probability of both $x$ and $y$ being true simultaneously (according to some [https://arbital.com/p/-probability_distribution](https://arbital.com/p/-probability_distribution) $\mathbb P$). $\mathbb P(x\mid y)$, pronounced "the conditional probability of x, given y", is defined to be the quantity

$$\frac{\mathbb P(x \wedge y)}{\mathbb P(y)}.$$

For example, in the [Diseasitis](https://arbital.com/p/22s) problem, $\mathbb P({sick}\mid {positive})$ is the probability that a patient is sick _given_ a positive test result, and it's calculated by taking the 18% patients who are sick *and* have positive test results, and dividing by all 42% of the patients who got positive test results.  That is, $\mathbb P({sick}\mid {positive})$ $=$ $\frac{\mathbb P({sick} \land {positive})}{\mathbb P({positive})}.$

Using a [frequency diagram](https://arbital.com/p/bayes_frequency_diagrams), we can visualize $\mathbb P(sick \mid positive)$ as the probability of drawing a $sick$ result from a bag of only those people in the population who got a $positive$ result.

![diseasitis frequency](https://i.imgur.com/0XDbrYi.png?0)

![bag of 18 and 24 patients](https://i.imgur.com/W6avfHO.png?0)

The "given" operator in $\mathbb P(x\mid y)$ tells us to assume that $y$ is true, to restrict our attention to only possible cases where $y$ is true, and then ask about the probability of $x$ *within* those cases.

Note that $\mathbb P(positive \mid sick)$ is _not_ the same as $\mathbb P(sick \mid positive).$ To find the probability that a patient has a positive result given that they're sick, we can visualize taking the 20 sick patients and putting them in a group, and then asking the probability that a randomly selected one will have a positive result, which will be $\frac{18}{20} = 0.9$ — so $\mathbb P(positive \mid sick) = 90\%,$ while $\mathbb P(sick \mid positive) \approx 43\%.$ Mixing up which one is which is an unfortunate source of of many practical errors when you're trying to do these calculations using only the formal notation, at least until you get used to it. Just remember that $\mathbb P(\text{left} \mid \text{right})$ is the probability of the thing on the left _given_ that the thing on the right is true.

Adom Hartell
Eric Bruylant
Nate Soares
Cameron McFee


The n-th root of m is either an integer or irrational

There is an intuitive way to see that for any natural numbers $m$ and $n$, $\sqrt[https://arbital.com/p/n](https://arbital.com/p/n)m$ will always either be an integer or an irrational number.

Suppose that there was some $\sqrt[https://arbital.com/p/n](https://arbital.com/p/n)m$ that was a rational number $\frac{a}{b}$ that was not an integer. Suppose further that $\frac ab$ is written as a [https://arbital.com/p/-reduced](https://arbital.com/p/-reduced) fraction, such that the [https://arbital.com/p/-greatest_common_divisor](https://arbital.com/p/-greatest_common_divisor) of $a$ and $b$ is $1$. Then, since $\frac{a}{b}$ is not an integer, $b > 1$.

Since $\frac ab = \sqrt[https://arbital.com/p/n](https://arbital.com/p/n)m$, we have conversely that $(\frac ab)^n = m$, which is a natural number by our hypothesis. But let's take a closer look at $(\frac ab)^n$. It evaluates to $\frac{a^n}{b^n}$, which is still a reduced fraction.

But since $b > 1$ before, we have that $b^n > 1$ as well, meaning that $(\frac ab)^n$ cannot be an integer, contradicting the fact that it equals $m$, a natural number.


Probability distribution: Motivated definition

When discussing probabilities, people will often (informally) say things like "well, the probability $\mathbb P(sick)$ of the patient being sick is about 20%." What does this $\mathbb P(sick)$ notation mean?

Intuitively, $\mathbb P(sick)$ is supposed to denote the probability that a particular person is sick (on a scale from 0 to 1). But how is $\mathbb P(sick)$ defined? Is there an objective probability of sickness? If not, where does the number come from?

At first you might be tempted to say $\mathbb P(sick)$ is defined by the surrounding population: If 1% of people are sick at any given time, then maybe $\mathbb P(sick)$ should be 1%. But what if this person is currently running a high fever and complaining about an upset stomach? Then we should probably assign a probability higher than 1%.

Next you might be tempted to say that the _true_ probability of the person being sick is either 0 or 1 (because they're either sick or they aren't), but this observation doesn't really help us manage our _own_ uncertainty. It's all well and good to say "either they sick or they aren't," but if you're a doctor who has to choose which medication to prescribe (and different ones have different drawbacks), then you need some way of talking about how sick they _seem_ to be (given what you've seen).

This leads us to the notion of [subjective probability](https://arbital.com/p/4vr). _Your_ probability that a person is sick is a fact about _you._ They are either sick or healthy, and as you observe more facts about them (such as "they're running a fever"), your _personal_ belief in their health vs sickness changes. This is the idea that used to define notation like $\mathbb P(sick).$

Formally, $\mathbb P(sick)$ is defined to be the probability that $\mathbb P$ assigns to $sick,$ where $\mathbb P$ is a type of object known as a "probability distribution", which is an object designed for keeping track of (and managing) uncertainty. Specifically, probability distributions are objects that distribute a finite amount of "stuff" across a large number of "states," and $\mathbb P(sick)$ measures how much stuff $\mathbb P$ _in particular_ puts on $sick$-type states. For example, the states could be cups with labels on them, and the stuff could be water, in which case $\mathbb P(sick)$ would be the proportion of all water in the $sick$-labeled cups.

The "stuff" and "states" may be arbitrary: you can build a probability distribution out of water in cups, clay in cubbyholes, abstract numbers represented in a computer, or weightings between neurons in your head. The stuff is called "probability mass," the states are called "possibilities."

To be even more concrete, imagine you build $\mathbb P$ out of cups and water, and that you give some of the cups suggestive labels like $sick$ and $healthy$. Then you can talk about the proportion of all probability-water that's in the $sick$ cup vs the $healthy$ cup. This is a probability distribution, but it's not a very useful one. In practice, we want to model more than one thing at a time. Let's say that you're a doctor at an immigration center who needs to assess a person's health, age, and country of origin. Now the set of possibilities that you want to represent aren't just $sick$ and $healthy,$ they're _all combinations_ of health, age, and origin:

$$
\begin{align}
sick, \text{age }1, \text{Afghanistan} \\
healthy, \text{age }1, \text{Afghanistan} \\
sick, \text{age }2, \text{Afghanistan} \\
\vdots \\
sick, \text{age }29, \text{Albania} \\
healthy, \text{age }29, \text{Albania} \\
sick, \text{age }30, \text{Albania} \\
\vdots
\end{align}
$$

and so on. If you build this probability distribution out of cups, you're going to need a lot of cups. If there are 2 possible health states ($sick$ and $healthy$), 150 possible ages, and 196 possible countries, then the total number of cups you need in order to build this probability distribution is $2 \cdot 150 \cdot 196 = 58800,$ which is rather excessive. (There's a reason we do probabilistic reasoning using transistors and/or neurons, as opposed to cups with water in them).

In order to make this proliferation of possibilities manageable, the possibilities are usually arranged into columns, such as the "Health", "Age", and "Country" columns above. This columns are known as "[variables](https://arbital.com/p/random_variable)" of the distribution. Then, $\mathbb P(sick)$ is an abbreviation for $\mathbb P(\text{Health}=sick),$ which counts the proportion of all probability mass (water) allocated to possibilities (cups) that have $sick$ in the Health column of their label.

What's the point of doing all this setup? Once we've made a probability distribution, we can hook it up to the outside world such that, when the world interacts with the probability distribution, the probability mass is shifted around inside the cups. For example, if you have a rule which says "whenever a person shows me a passport from country X, I throw out all water except the water in cups with X in the Country column", then, whenever you see a passport, the probability distribution will get more accurate.

The natural question here is, what are the best ways to manipulate the probability mass in $\mathbb P$ (in response to observations), if the goal is to have $\mathbb P$ get more and more accurate over time? That's exactly the sort of question that [https://arbital.com/p/-1bv](https://arbital.com/p/-1bv) can be used to answer (and it has implications both for artificial intelligence, and for understanding human intelligence — after all, _we ourselves_ are a physical system that manages uncertainty, and updates beliefs in response to observations).

At this point, there are two big objections to answer. First objection:

> Whoa now, the number of cups in $\mathbb P$ got pretty big pretty quickly, and this was a simple example. In a realistic probability distribution $\mathbb P$ intended to represent the real world (which has way more than 3 variables worth tracking), the number of necessary possibilities would be _ridiculous._ Why do we define probabilities in terms of these huge impractical "probability distributions"?

This is an important question, which is answered by three points:

1. In practice, there are a number of tricks for exploiting regularities in the structure of the world in order to drastically reduce the number of cups you need to track. We won't be covering those tricks in this guide, but you can check out [Realistic probabilities](https://arbital.com/p/) and [Arbital's guide to Bayes nets](https://arbital.com/p/) if you're interested in the topic.
2. Even so, full-fledged probabilistic reasoning _is_ computationally infeasible on complex problems. In practice, physical reasoning systems (such as brains or artificial intelligence algorithms) use lots of approximations and shortcuts.
3. Nevertheless, reasoning according to a full probability distribution is the _theoretical ideal_ for how to do good reasoning. [You can't do better than probabilistic reasoning](https://arbital.com/p/) (unless you're born knowing the right answers to everything), and [insofar as you don't use probabilistic reasoning, you can be exploited](https://arbital.com/p/). Even if complex probability distribution are too big to manage in practice, they tell give lots of hints about how to reason right or wrong that we can follow in our day-to-day lives.

Second objection:

> You basically just said "given a bunch of cups and some water, we define the probability of a person being sick as the amount of water in some suggestively-labeled cups." How does that have anything to do with whether or not the person is actually sick? Just because you put a $sick$ label on there doesn't magically give the water meaning!

This is an important point. For $\mathbb P$ to be useful, we want to design a reasoning procedure such that the more we interact with a person, the more probability mass starts to reflect how healthy the person actually is. That is, we want the water to go into $sick$ cups if they're sick, and $healthy$ cups if they're healthy. If our reasoning procedure has that property, and we have $\mathbb P$ interact with the world for a while, then its probabilities will get pretty accurate — at which point $\mathbb P$ can be used to answer questions and/or make decisions. (This is the principle that makes brains and artificial intelligence algorithms tick.)

How do we design reasoning mechanisms that cause $\mathbb P$ to become more accurate the more it interacts with the world? That's a [big question](https://arbital.com/p/how_to_get_accurate_probabilities), and the answer has many parts. One of the most important parts of the answer, though, is a law of probability theory which tells us the correct way to move the probability mass around in response to new observations (assuming the goal is to make $\mathbb P$ more accurate). For more on that law, see [Bayes' rule](https://arbital.com/p/1lz).


Math 3 example statements

If you're at a Math 3 level, you'll probably be familiar with at least some of these sentences and formulas, or you would be able to understand what they meant on a surface level if you were to look them up. Note that you don't necessarily have to understand the *proofs* of these statements (that's what we're here for, to teach you what they mean), but your eyes shouldn't gloss over them either.

> In a [group](https://arbital.com/p/3gd) $G$, the [https://arbital.com/p/-4bj](https://arbital.com/p/-4bj) of an element $g$ is the set of elements that can be written as $hgh^{-1}$ for all $h \in G$.

> The [rank-nullity theorem](https://arbital.com/p/) states that for any [https://arbital.com/p/-linear_mapping](https://arbital.com/p/-linear_mapping) $f: V \to W$, the [https://arbital.com/p/-dimension](https://arbital.com/p/-dimension) of the [https://arbital.com/p/-image](https://arbital.com/p/-image) of $f$ plus the dimension of the [https://arbital.com/p/-kernel](https://arbital.com/p/-kernel) of $f$ is equal to the dimension of $V$.

> A [Baire space](https://arbital.com/p/) is a space that satisfies [Baire's Theorem](https://arbital.com/p/) on [complete metric spaces](https://arbital.com/p/complete_metric_space): For a [https://arbital.com/p/-topological_space](https://arbital.com/p/-topological_space) $X$, if ${F_1, F_2, F_3, \ldots}$ is a [countable](https://arbital.com/p/countable_set) collection of open sets that are [dense](https://arbital.com/p/dense_set) in $X$, then $\bigcap_{n=1}^\infty F_n$ is also dense in $X$.

> The [https://arbital.com/p/riemann_hypothesis](https://arbital.com/p/riemann_hypothesis) asserts that every non-trivial zero of the [https://arbital.com/p/riemann_zeta_function](https://arbital.com/p/riemann_zeta_function) $\zeta(s) = \sum_{n=1}^\infty \frac{1}{s^n}$ when $s$ is a complex number has a real part equal to $\frac12$.

> $\newcommand{\pd}[https://arbital.com/p/2](https://arbital.com/p/2){\frac{\partial #1}{\partial #2}}$ The [https://arbital.com/p/jacobian_matrix](https://arbital.com/p/jacobian_matrix) of a [vector-valued function](https://arbital.com/p/vector_valued_function) $f: \mathbb{R}^m \to \mathbb{R}^n$ is the matrix of [https://arbital.com/p/-partial_derivatives](https://arbital.com/p/-partial_derivatives) $\left[\begin{matrix} \pd{y_1}{x_1} & \pd{y_1}{x_2} & \cdots & \pd{y_1}{x_m} \\ \pd{y_2}{x_1} & \pd{y_2}{x_2} & \cdots & \pd{y_2}{x_m} \\ \vdots & \vdots & \ddots & \vdots \\ \pd{y_n}{x_1} & \pd{y_n}{x_2} & \cdots & \pd{y_n}{x_m} \end{matrix} \right](https://arbital.com/p/)$ between each component of the argument vector $x = (x_1, x_2, \ldots, x_m)$ and each component of the result vector $y = f(x) = (y_1, y_2, \ldots, y_n)$. It is notated as $\displaystyle \frac{d\mathbf{y}}{d\mathbf{x}}$ or $\displaystyle \frac{d(y_1, y_2, \ldots, y_n)}{d(x_1, x_2, \ldots, x_m)}$.


Proof of Bayes' rule: Probability form

Let $\mathbf H$ be a [variable](https://arbital.com/p/random_variable) in $\mathbb P$ for the true hypothesis, and let $H_k$ be the possible values of $\mathbf H,$ such that $H_k$ is [https://arbital.com/p/-1rd](https://arbital.com/p/-1rd). Then, Bayes' theorem states:

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e\mid H_i) \cdot \mathbb P(H_i)}{\sum_k \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)},$$

with a proof that runs as follows. By the definition of [https://arbital.com/p/-1rj](https://arbital.com/p/-1rj),

$$\mathbb P(H_i\mid e) = \dfrac{\mathbb P(e \wedge H_i)}{\mathbb P(e)} = \dfrac{\mathbb P(e \mid  H_i) \cdot \mathbb P(H_i)}{\mathbb P(e)}$$

By the law of [marginal probability](https://arbital.com/p/law_of_marginal_probability):

$$\mathbb P(e) = \sum_{k} \mathbb P(e \wedge H_k)$$

By the definition of conditional probability again:

$$\mathbb P(e \wedge H_k) = \mathbb P(e\mid H_k) \cdot \mathbb P(H_k)$$

Done.

Note that this proof of Bayes' rule is less general than the [proof](https://arbital.com/p/1xr) of the [odds form of Bayes' rule](https://arbital.com/p/1x5).

## Example

Using the [Diseasitis](https://arbital.com/p/22s) example problem, this proof runs as follows:

$$\begin{array}{c}
\mathbb P({sick}\mid {positive}) = \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive})} \\[https://arbital.com/p/0.3em](https://arbital.com/p/0.3em)
= \dfrac{\mathbb P({positive} \wedge {sick})}{\mathbb P({positive} \wedge {sick}) + \mathbb P({positive} \wedge \neg {sick})} \\[https://arbital.com/p/0.3em](https://arbital.com/p/0.3em)
= \dfrac{\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})}{(\mathbb P({positive}\mid {sick}) \cdot \mathbb P({sick})) + (\mathbb P({positive}\mid \neg {sick}) \cdot \mathbb P(\neg {sick}))}
\end{array}
$$

Numerically:

$$3/7 = \dfrac{0.18}{0.42} = \dfrac{0.18}{0.18 + 0.24} = \dfrac{90\% * 20\%}{(90\% * 20\%) + (30\% * 80\%)}$$

Using red for sick, blue for healthy, and + signs for positive test results, the proof above can be visually depicted as follows:

![bayes theorem probability](https://i.imgur.com/H9im04o.png?0)

%todo: if we replace the other Venn diagram for the proof of Bayes' rule, we should probably update this one too.%

Eric Bruylant
Nate Soares
Eliezer Yudkowsky
Alexei Andreev


Math 2 example statements

If you're at a Math 2 level, you'll probably be familiar with most or all of these sentences and formulas, or you would be able to understand what they meant on a surface level if you were to look them up.

> The [https://arbital.com/p/-quadratic_formula](https://arbital.com/p/-quadratic_formula) states that the roots of every quadratic expression $ax^2 + bx + c$ are equal to $\displaystyle \frac{-b \pm \sqrt{b^2 - 4ac}}{2a}$. The expression under the [https://arbital.com/p/-square_root](https://arbital.com/p/-square_root), $b^2 - 4ac$, is called the [https://arbital.com/p/-discriminant](https://arbital.com/p/-discriminant), and determines how many roots there are in the equation.

> The imaginary number $i$ is defined as the primary root of the quadratic equation $x^2 + 1 = 0$.

> To solve the system of linear equations $\begin{matrix}ax + by = c \\ dx + ey = f\end{matrix}$ for $x$ and $y$, the value of $x$ can be computed as $\displaystyle \frac{bf - ce}{bd - ae}$.

> The [https://arbital.com/p/-power_rule](https://arbital.com/p/-power_rule) in calculus states that $\frac{d}{dx} x^n = nx^{n-1}$.

> All the solutions to the equation $m^n = n^m$ where $m < n$ are of the form $m = (1 + \frac 1x)^x$ and $n = (1 + \frac 1x)^{x+1}$, where $x$ is any positive real number.


Binary notation

When you were first taught how to write [numbers](https://arbital.com/p/54y) and [add](https://arbital.com/p/addition) them together, you were probably told something about the "ones place", the "tens place", the "hundreds place", and so on.  Each [https://arbital.com/p/-digit](https://arbital.com/p/-digit) further to the left represents a larger [multiple](https://arbital.com/p/math_power) of ten, and the multiples are added together to get the number you want--so $8207$ is counted as $(7 \times 10^0) + (0 \times 10^1) + (2 \times 10^2) + (8 \times 10^3)$, or "seven ones, zero tens, two hundreds, and eight thousands".  But why use powers of ten?  What's so special about that number?

Well, nothing, actually. We probably only use powers of ten because we happen to have evolved with ten fingers--any number bigger than one will work just as well, although it may look strange at first.  Since two is the smallest [https://arbital.com/p/-48l](https://arbital.com/p/-48l) larger than one, a *binary* number notation--a notation that uses powers of two--is one of the simplest possible. %note:The word "binary" has the same root as the word "bicycle"--*bi* meaning "two".%

Binary notation uses only two digits, $0$ and $1$, and each "place" to the left goes up by a power of two instead of ten (in other words, it doubles).  For example, in binary notation, the number $11010$ is counted as $(0 \times 2^0) + (1 \times 2^1) + (0 \times 2^2) + (1 \times 2^3) + (1 \times 2^4)$, or "zero ones, one two, zero fours, one eight, and one sixteen".  Translating back to the familiar [base](https://arbital.com/p/number_bases) ten, we would write it as $26$.

You may notice that binary notation tends to be a bit longer than [decimal](https://arbital.com/p/4sl) (for example, "11010" takes more characters to write than "26").  It's also more difficult to read, unless you have a lot of practice with it.  So why would anyone use it?  Well, for one thing, binary notation is often very convenient for talking about powers of two, for instance when working with base-two [logarithms](https://arbital.com/p/3nd) or log [https://arbital.com/p/-1rb](https://arbital.com/p/-1rb), or when working with some quantity measured in [bits](https://arbital.com/p/3y2).  It's also essential for working with computers, as all modern computers store and manipulate data, on the lowest level, exclusively using binary notation.

Malcolm McCrimmon
Eric Bruylant
Eric Rogstad

Binary notation is a way to write [numbers](https://arbital.com/p/54y) using [multiples](https://arbital.com/p/math_power) of two instead of the usual multiples of ten. It's especially useful for working with computers, which use binary notation exclusively.


Likelihood function

Let's say you have a piece of evidence $e$ and a set of hypotheses $\mathcal H.$ Each $H_i \in \mathcal H$ assigns some [likelihood](https://arbital.com/p/56v) to $e.$ The function $\mathcal L_{e}(H_i)$ that reports this likelihood for each $H_i \in \mathcal H$ is known as a "likelihood function."

For example, let's say that the evidence is $e_c$ = "Mr. Boddy was killed with a candlestick," and the hypotheses are $H_S$ = "Miss Scarlett did it," $H_M$ = "Colonel Mustard did it," and $H_P$ = "Mrs. Peacock did it." Furthermore, if Miss Scarlett was the murderer, she's 20% likely to have used a candlestick. By contrast, if Colonel Mustard did it, he's 10% likely to have used a candlestick, and if Mrs. Peacock did it, she's only 1% likely to have used a candlestick. In this case, the likelihood function is

$$\mathcal L_{e_c}(h) =
\begin{cases}
0.2 & \text{if $h = H_S$} \\
0.1 & \text{if $h = H_M$} \\
0.01 & \text{if $h = H_P$} \\
\end{cases}
$$

For an example using a continuous function, consider a possibly-biased coin whose bias $b$ to come up heads on any particular coinflip might be anywhere between $0$ and $1$.  Suppose we observe the coin to come up heads, tails, and tails. We will denote this evidence $e_{HTT}.$ The likelihood function over each hypothesis $H_b$ =  "the coin is biased to come up heads $b$ portion of the time" for $b \in [1](https://arbital.com/p/0,)$ is:

$$\mathcal L_{e_{HTT}}(H_b) = b\cdot (1-b)\cdot (1-b).$$

There's no reason to [normalize](https://arbital.com/p/1rk) likelihood functions so that they sum to 1 — they aren't probability distributions, they're functions expressing each hypothesis' propensity to yield the observed evidence. For example, if the evidence was really obvious ($e_s$ = "the sun rose this morning,") it might be the case that almost all hypotheses have a very high likelihood, in which case the sum of the likelihood function will be much more than 1.

Likelihood functions carry _absolute_ likelihood information, and therefore, they contain information that [relative likelihoods](https://arbital.com/p/1rq) do not. Namely, absolute likelihoods can be used to check a hypothesis for [strict confusion](https://arbital.com/p/227).


Likelihood ratio

Given a piece of evidence $e_0$ and two hypothsese $H_i$ and $H_j,$ the likelihood ratio between them is the ratio of the [https://arbital.com/p/-56v](https://arbital.com/p/-56v) each hypothesis assigns to $e_0.$

For example, imagine the evidence is $e$ = "Mr. Boddy was knifed", and the hypotheses are $H_P$ = "Professor Plum did it" and $H_W$ = "Mrs. White did it." Let's say that, if Professor Plum were the killer, we're 25% sure he would have used a knife. Let's also say that, if Mrs. White were the killer, there's only a 5% chance she would have used a knife. Then the likelihood ratio of $e_0$ between $H_P$ and $H_W$ is 25/5 = 5, which says that $H_P$ assigns five times as much likelihood to $e$ as does $H_W,$ which means that the evidence supports the "Plum did it" hypothesis five times as much as it supports the "Mrs. White did it" hypothesis.

A likelihood ratio of 5 denotes [relative likelihoods](https://arbital.com/p/1rq) of $(5 : 1).$ Relative likelihoods can be multiplied by [odds](https://arbital.com/p/1rb) in order to update those odds, as per [https://arbital.com/p/1lz](https://arbital.com/p/1lz).

Given a piece of evidence $e$ and two hypothsese $H_i$ and $H_j,$ the likelihood ratio between them is the ratio of the [https://arbital.com/p/-56v](https://arbital.com/p/-56v) each hypothesis assigns to $e$ For example, let $e$ = "Mr. Boddy was knifed", and say that Professor Plum is 25% likely to use a knife while Mrs. White is only 5% likely to use a knife. Then the likelihood ratio of $e$ between the hypotheses "Plum did it" and "Mrs. White did it" is 25/5 = 5/1. See also [https://arbital.com/p/1rq](https://arbital.com/p/1rq).


Likelihood

Consider a piece of evidence $e,$ such as "Mr. Boddy was shot." We might have a number of different hypotheses that explain this evidence, including $H_S$ = "Miss Scarlett killed him", $H_M$ = "Colonel Mustard killed him", and so on.

Each of those hypotheses assigns a different probability to the evidence. For example, imagine that _if_ Miss Scarlett _were_ the killer, there's a 20% chance she would use a gun, and an 80% chance she'd use some other weapon. In this case, the "Miss Scarlett" hypothesis assigns a *likelihood* of 20% to $e.$

When reasoning about different hypotheses using a [probability distribution](https://arbital.com/p/-probability_distribution) $\mathbb P$, the likelihood of evidence $e$ given hypothesis $H_i$ is often written using the [conditional probability](https://arbital.com/p/1rj) $\mathbb P(e \mid H_i).$ When reporting likelihoods of many different hypotheses at once, it is common to use a [https://arbital.com/p/-likelihood_function,](https://arbital.com/p/-likelihood_function,) sometimes written [$\mathcal L_e](https://arbital.com/p/51n).

[Relative likelihoods](https://arbital.com/p/1rq) measure the degree of support that a piece of evidence $e$ provides for different hypotheses. For example, let's say that if Colonel Mustard were the killer, there's a 40% chance he would use a gun. Then the absolute likelihoods of $H_S$ and $H_M$ are 20% and 40%, for _relative_ likelihoods of (1 : 2). This says that the evidence $e$ supports $H_M$ twice as much as it supports $H_S,$ and that the amount of support would have been the same if the absolute likelihoods were 2% and 4% instead.

According to [Bayes' rule](https://arbital.com/p/1lz), relative likelihoods are the appropriate tool for measuring the [strength of a given piece evidence](https://arbital.com/p/22x). Relative likelihoods are one of two key constituents of belief in [Bayesian reasoning](https://arbital.com/p/bayesian_reasoning), the other being [prior probabilities](https://arbital.com/p/1rm).

While absolute likelihoods aren't necessary when updating beliefs by Bayes' rule, they are useful when checking for [confusion](https://arbital.com/p/227). For example, say you have a coin and only two hypotheses about how it works: $H_{0.3}$ = "the coin is random and comes up heads 30% of the time", and $H_{0.9}$ = "the coin is random and comes up heads 90% of the time." Now let's say you toss the coin 100 times, and observe the data HTHTHTHTHTHTHTHT... (alternating heads and tails). The _relative_ likelihoods strongly favor $H_{0.3},$ because it was less wrong. However, the _absolute_ likelihood of $H_{0.3}$ will be much lower than expected, and this deficit is a hint that $H_{0.3}$ isn't right. (For more on this idea, see [https://arbital.com/p/227](https://arbital.com/p/227).)

"Likelihood", when speaking of Bayesian reasoning, denotes *the probability of an observation, supposing some hypothesis to be correct.*

Suppose our piece of evidence $e$ is that "Mr. Boddy was shot."  One of our suspects is Miss Scarlett, and we denote by $H_S$ the hypothesis that Miss Scarlett shot Mr. Boddy.  Suppose that if Miss Scarlett *were* the killer, we'd have predicted in advance a 20% probability she would use a gun, and an 80% chance she'd use some other weapon.

Then the *likelihood* from the evidence, to Miss Scarlett being the killer, is 0.20.  Using [conditional probability notation](https://arbital.com/p/1rj), $\mathbb P(e \mid H_S) = 0.20.$

This doesn't mean Miss Scarlett has a 20% chance of being the killer; it means that if she is the killer, our observation had a probability of 20%.

Relative likelihoods are a key ingredient for [Bayesian reasoning](https://arbital.com/p/1ly) and one of the quantities plugged into [Bayes's Rule](https://arbital.com/p/1lz).


Subtraction of rational numbers (Math 0)

So far, we have met [the idea of a rational number](https://arbital.com/p/4zx), treating them as chunks of apples, and [how to add them together](https://arbital.com/p/55m).
Now we will discover how the idea of the anti-apple (by analogy with the [integers' anti-cow](https://arbital.com/p/53r)) must work.

# The anti-apple

Just as we had an anti-cow, so we can have an anti-apple.
If we combine an apple with an anti-apple, they both annihilate, leaving nothing behind.
We write this as $1 + (-1) = 0$.

A very useful thing for you to ponder for thirty seconds (though I will give you the answer soon): given that $\frac{a}{n}$ means "divide an apple into $n$ equal pieces, then take $a$ copies of the resulting little-piece", what would $\frac{-1}{n}$ mean? And what would $-\frac{1}{n}$ mean?

%%hidden(Show possible solution):
$\frac{-1}n$ would mean "divide an apple into $n$ equal pieces, then take $-1$ copies of the resulting little-piece".
That is, turn it into an anti-little-piece.
This anti-little-piece will annihilate one little-piece of the same size: $\frac{-1}{n} + \frac{1}{n} = 0$.

$-\frac{1}{n}$, on the other hand, would mean "divide an anti-apple into $n$ equal pieces, then take $1$ copy of the resulting little-anti-piece".
But this is the same as $\frac{-1}{n}$: it doesn't matter whether we do "convert to anti, then divide up the apple" or "divide up the apple, then convert to anti".
That is, "little-anti-piece" is the same as "anti-little-piece", which is very convenient.
%%

What about chunks of apple?
If we combine half an apple with half an anti-apple, they should also annihilate, leaving nothing behind.
We write this as $\frac{1}{2} + \left(-\frac{1}{2}\right) = 0$.

How about a bit more abstract?
If we combine an apple with half an anti-apple, what should happen?
Well, the apple can be made out of two half-chunks (that is, $1 = \frac{1}{2} + \frac{1}{2}$); and we've just seen that half an apple will annihilate half an anti-apple; so we'll be left with just one of the two halves of the apple.
More formally, $1 + \left(-\frac{1}{2}\right) = \frac{1}{2}$; or, writing out the calculation in full, $$1 + \left(-\frac{1}{2}\right) = \frac{1}{2} + \frac{1}{2} + \left(-\frac{1}{2}\right) = \frac{1}{2}$$

Let's go the other way round: if we combine an anti-apple with half an apple, what happens?
It's pretty much the same as the opposite case except flipped around: the anti-apple is made of two anti-half-chunks, and the half apple will annihilate one of those chunks, leaving us with half an anti-apple: that is, $(-1) + \frac{1}{2} = -\frac{1}{2}$.

We call all of these things **subtraction**: "subtracting" a quantity is defined to be the same as the addition of an anti-quantity.

# General procedure

Since we already know how to add, we might hope that subtraction will be easier (since subtraction is just a slightly different kind of adding).

In general, we write $-\frac{1}{n}$ for "the $\frac{1}{n}$-sized building block, but made by dividing an *anti*-apple (instead of an apple) into $n$ equal pieces".
Remember from the pondering above that this is actually the same as $\frac{-1}{n}$, where we have divided an apple into $n$ equal pieces but then taken $-1$ of the pieces.

Then in general, we can just use the instant addition rule that we've already seen. %%note:Recall: this was $\frac{a}{m} + \frac{b}{n} = \frac{a\times n + b \times m}{m \times n}$. Remember that the order of operations in the integers is such that in the numerator, we calculate both the products first; then we add them together.%%
In fact, $$\frac{a}{m} - \frac{b}{n} = \frac{a}{m} + \left(\frac{-b}{n}\right) = \frac{a \times n + (-b) \times m}{m \times n} = \frac{a \times n - b \times m}{m \times n}$$

Why are we justified in just plugging these numbers into the formula, without justification?
You're quite right if you are dubious: you should not be content merely to *learn* this formula%%note:This goes for all of maths! It's not simply a collection of arbitrary rules, but a proper process that we use to model our thoughts. Behind every pithy, unmemorable formula is a great edifice of motivation and reason, if you can only find it.%%.
In the rest of this page, we'll go through why it works, and how you might construct it yourself if you forgot it.
I took the choice here to present the formula first, because it's a good advertisement for why we use the $\frac{a}{n}$ notation rather than talking about "$\frac{1}{n}$-chunks" explicitly: it's a very compact and neat way of expressing all this talk of anti-apples, in the light of what we've already seen about addition.

Very well: what should $\frac{a}{m} - \frac{b}{n}$ be?
We should first find a smaller chunk out of which we can build both the $\frac{1}{m}$ and $\frac{1}{n}$ chunks.
We've seen already that $\frac{1}{m \times n}$ will work as a smaller chunk-size.

Now, what is $\frac{a}{m}$ expressed in $\frac{1}{m \times n}$-chunks?
Each $\frac{1}{m}$-chunk is $n$ of the $\frac{1}{m \times n}$-chunks, so $a$ of the $\frac{1}{m}$-chunks is $a$ lots of "$n$ lots of $\frac{1}{m \times n}$-chunks": that is, $a \times n$ of them.

Similarly, $\frac{b}{n}$ is just $b \times m$ lots of $\frac{1}{m \times n}$-chunks.

So, expressed in $\frac{1}{m \times n}$-chunks, we have $a \times n$ lots of positive chunks, and $b \times m$ lots of anti-chunks.
Therefore, when we put them together, we'll get $a \times n - b \times m$ chunks (which might be negative or positive or even zero - after all the annihilation has taken place we might end up with either normal or anti-chunks or maybe no chunks at all - but it's still an integer, being a number of chunks).

So the total amount of apple we have is $\frac{a \times n - b \times m}{m \times n}$, just like we got out of the instant formula.

Subtraction is "adding anti-apples", or equivalently removing some apples you already have.


Transitive relation

A binary [https://arbital.com/p/-3nt](https://arbital.com/p/-3nt) $R$ is **transitive** if whenever $aRb$ and $bRc$, $aRc$.

The most common examples or transitive relations are [partial orders](https://arbital.com/p/3rb) (if $a \leq b$ and $b \leq c$, then $a \leq c$) and [equivalence relations](https://arbital.com/p/53y) (if $a \sim b$ and $b \sim c$, then $a \sim c$).

A transitive relation that is also [reflexive](https://arbital.com/p/5dy) is called a [https://arbital.com/p/-preorder](https://arbital.com/p/-preorder).

A [https://arbital.com/p/-transitive_set](https://arbital.com/p/-transitive_set) $S$ is a set on which the element-of relation $\in$ is transitive; whenever $a \in x$ and $x \in S$, $a \in S$.

Dylan Hendrickson
Martin Epstein
Joe Zeng


Lattice: Examples

Here are some additional examples of lattices. $\newcommand{\nsubg}{\mathcal N \mbox{-} Sub~G}$

A familiar example
---------------------------------

Consider the following lattice.

![Suspicious Lattice Hasse Diagram](http://i.imgur.com/7HefDKm.png)

Does this lattice look at all familiar to you? From some other area of mathematics, perhaps?

%%hidden(Reveal the truth):

In fact, this lattice corresponds to boolean logic, as can be seen when we replace b with true and a with false in the following "truth table".

![lattice truth table](http://i.imgur.com/hpThsTk.png)

%%comment:

Latex source:

\begin{tabular} {| c | c | c | c |}
  \hline
  $x$ & $y$ & $x \vee y$ & $x \wedge y$ \\ \hline
  $a$ & $a$ & $a$ & $a$ \\ \hline
  $a$ & $b$ & $b$ & $a$ \\ \hline
  $b$ & $a$ & $b$ & $a$ \\ \hline
  $b$ & $b$ & $b$ & $b$ \\ \hline
\end{tabular}

%%

%%

Normal subgroups
---------------------------------

Let $G$ be a group, and let $\nsubg$ be the set of all [normal subgroups](https://arbital.com/p/4h6) of $G$. Then $\langle \nsubg, \subseteq \rangle$ is a lattice where for $H, K \in \nsubg$, $H \wedge K = H \cap K$, and $H \vee K = HK = \{ hk \mid h \in H, k \in K \}$.

%%hidden(Proof):

Let $H,K \in \nsubg$. Then $H \wedge K = H \cap K$. We first note that $H \cap K$ is a [subgroup](https://arbital.com/p/576) of $G$. For let $a,b \in H \cap K$. Since $H$ is a group, $a \in H$, and $b \in H$, we have $ab \in H$. Likewise, $ab \in K$. Combining these, we have $ab \in H \cap K$, and so $H \cap K$ is satisfies the closure requirement for subgroups. Since $H$ and $K$ are groups, $a \in H$, and $a \in K$, we have $a^{-1} \in H$ and $a^{-1} \in K$. Hence, $a^{-1} \in H \cap K$, and so $H \cap K$ satisfies the inverses requirement for subgroups. Since $H$ and $K$ are subgroups of $G$, we have $e \in H$ and $e \in K$. Hence, we have $e \in H \cap K$, and so $H \cap K$ satisfies the identity requirement for subgroups.

Furthermore, $H \cap K$ is a normal subgroup, because for all $a \in G$, $a^{-1}(H \cap K)a = a^{-1}Ha \cap a^{-1}Ka = H \cap K$. It's clear from the definition of intersection that $H$ and $K$ do not share a common subset larger than $H \cap K$.

For $H, K \in \nsubg$, we have $H \vee K = HK = \{ hk \mid h \in H, k \in K \}$.

First we will show that $HK$ is a group. For $hk, h'k' \in HK$, since $kH = Hk$, there is some $h'' \in H$ such that $kh' = h''k$. Hence, $hkh'k' = hh''kk' \in HK$, and so $HK$ is closed under $G$'s group action. For $hk \in HK$, we have $(hk)^{-1} = k^{-1}h^{-1} \in k^{-1}H = Hk^{-1} \subseteq HK$, and so $HK$ is closed under inversion. Since $e \in H$ and $e \in K$, we have $e = ee \in HK$. Finally, $HK$ inherits its associativity from $G$.

To see that $HK$ is a normal subgroup of $G$, let $a \in G$. Then $a^{-1}HKa = Ha^{-1}Ka = HKa^{-1}a = HK$.

There is no subgroup $F$ of $G$ smaller than $HK$ which contains both $H$ and $K$. If there were such a subgroup, there would exist some $h \in H$ and some $k \in K$ such that $hk \not\in F$. But $h \in F$ and $k \in F$, and so from $F$'s group closure we conclude $hk \in F$, a contradiction.

%%


Subgroup

A **subgroup** of a [https://arbital.com/p/-3gd](https://arbital.com/p/-3gd) $(G,*)$ is a group of the form $(H,*)$, where $H \subset G$. We usually say simply that $H$ is a subgroup of $G$.

For a subset of a group $G$ to be a subgroup, it needs to satisfy all of the group axioms itself: [closure](https://arbital.com/p/3gy), [associativity](https://arbital.com/p/3h4), [identity](https://arbital.com/p/54p), and [inverse](https://arbital.com/p/inverse_element). We get associativity for free because $G$ is a group. So the requirements of a subgroup $H$ are:

1. **[Closure](https://arbital.com/p/3gy):** For any $x, y$ in $H$, $x*y$ is in $H$.
2. **[Identity](https://arbital.com/p/54p):** The identity $e$ of $G$ is in $H$.
3.  **[Inverses](https://arbital.com/p/inverse_element):** For any $x$ in $H$, $x^{-1}$ is also in $H$.

A subgroup is called [normal](https://arbital.com/p/4h6) if it is closed under [conjugation](https://arbital.com/p/4gk).

The subgroup [https://arbital.com/p/-3nt](https://arbital.com/p/-3nt) is [transitive](https://arbital.com/p/573): if $H$ is a subgroup of $G$, and $I$ is a subgroup of $H$, then $I$ is a subgroup of $G$.

#Examples

Any group is a subgroup of itself. The [https://arbital.com/p/-trivial_group](https://arbital.com/p/-trivial_group) is a subgroup of every group.

For any [https://arbital.com/p/-48l](https://arbital.com/p/-48l) $n$, the set of multiples of $n$ is a subgroup of the integers (under [https://arbital.com/p/-addition](https://arbital.com/p/-addition)).


Boolean

In formal [https://arbital.com/p/-258](https://arbital.com/p/-258), a boolean variable is a [https://arbital.com/p/-variable](https://arbital.com/p/-variable) that can take on one of only two possible values: "[https://arbital.com/p/-true](https://arbital.com/p/-true)" or "[https://arbital.com/p/-false](https://arbital.com/p/-false)".  [https://arbital.com/p/2v8](https://arbital.com/p/2v8) can then be said to [evaluate](https://arbital.com/p/-evaluation) to one of these two values, in the same way that ordinary [algebraic](https://arbital.com/p/algebra) expressions evaluate to a [https://arbital.com/p/-54y](https://arbital.com/p/-54y).

Also as in algebraic expressions, boolean values can be manipulated using certain operators such as [$\land$ ](https://arbital.com/p/and_operator), [$\lor$ ](https://arbital.com/p/or_operator), [negation_operator $\neg$ ([https://arbital.com/p/negation](https://arbital.com/p/negation), and [$\rightarrow$ ](https://arbital.com/p/implication_operator).  This field is called, surprisingly, [https://arbital.com/p/-boolean_algebra](https://arbital.com/p/-boolean_algebra).

Because booleans can only express [absolute truth or falsity](https://arbital.com/p/4mq), when working with measures of uncertainty you must use other representations, such as [https://arbital.com/p/-1rf](https://arbital.com/p/-1rf).

A boolean is a mathematical quantity that can only take on one of two values: "[https://arbital.com/p/-true](https://arbital.com/p/-true)" or "[https://arbital.com/p/-false](https://arbital.com/p/-false)". Booleans are to [https://arbital.com/p/-258](https://arbital.com/p/-258) what [integers](https://arbital.com/p/48l) are to [https://arbital.com/p/-arithmetic](https://arbital.com/p/-arithmetic).


Disambiguation

Several distinct concepts with comparable importance use this page's name, this page helps readers find what they're looking for.


Proof of Löb's theorem

The intuition behind the proof of Löb's theorem invokes the formalization of the following argument:

Let $S$ stand for the sentence "If $S$ is provable, then I am Santa Claus".

As it is standard, when trying to prove a If...then theorem we can start by supposing the antecedent and checking that the consequent follows. But if $S$ is provable, then we can chain its proof to an application of modus ponens of $S$ to itself in order to get a proof of the consequent "I am Santa Claus". If we suppose that showing the existence of a proof of a sentence implies that the sentence is true, then "I am Santa Claus" is true.

Thus we have supposed that $S$ is provably true then it follows that "I am Santa Claus", which is a proof of what $S$ states! Therefore, $S$ is provably true, and we can apply the same reasoning again (this time without supposing a counterfactual) to get that it is true that "I am Santa Claus".

Holy Gödel! Have we broken logic? If this argument worked, then we could prove any nonsensical thing we wanted. So where does it fail?

One suspicious point is our implicit assumption that $S$ can be constructed in the first place as a first order sentence. But we know that there exists a [provability predicate](https://arbital.com/p/5j7) encoding the meaning we need, and then we can apply the [diagonal lemma](https://arbital.com/p/59c) to the formula $Prv(x)\implies$ "I am Santa Claus" to get our desired $S$.

(We will ignore the fact that expressing "I am Santa Claus" in the language of first order logic is rather difficult, since we could easily replace it with something simple like $0=1$.)

It turns out that the culprit of this paradox is the apparently innocent supposition that when "I am Santa Claus" is provable, then it is true.

How can this be false?, you may ask. Well, the standard provability predicate $Prv$ is something of the form $\exists y Proof(x,y)$, where $Proof(x,y)$ is true iff $y$ encodes a valid proof of $x$. But $y$ might be a [nonstandard number](https://arbital.com/p/) encoding a [nonstandard proof](https://arbital.com/p/) of infinitely many steps. If the only proofs of $x$ are nonstandard then certainly you will never be able to prove $x$ from the axioms of the system using standard methods.

We can restate this result as Löb's theorem: If $Prv(A)\implies A$ is provable, then $A$ is provable.

-----
Now we go for the formal proof.

Let $T$ be an [axiomatizable](https://arbital.com/p/) extension of [minimal arithmetic](https://arbital.com/p/), and $P$ a one-place predicate satisfying the [Bernais-Hilbert derivability conditions](https://arbital.com/p/5j7), which are:

1. If $T\vdash A$, then $T\vdash P(A)$.
2. $T\vdash P(A\implies B) \implies (P(A)\implies P(B))$
3. $T\vdash P(A)\implies P(P(A))$.

For example, $PA$ and the standard provability predicate [satisfy those conditions](https://arbital.com/p/).

Let us suppose that $A$ is such that $T\vdash P(A)\implies A$.

As $T$ extends [minimal arithmetic](https://arbital.com/p/), the [https://arbital.com/p/-59c](https://arbital.com/p/-59c) is applicable, and thus there exists $S$ such that $T\vdash S \iff (P(S)\implies A)$.

By condition 1, $T\vdash P(S \implies (P(S)\implies A))$.

By condition 2, $T\vdash P(S) \implies P(P(S) \implies A)$.

Also by condition 2, $T\vdash P(P(S) \implies A) \implies (P(P(S)) \implies P(A))$.

Combining these, $T\vdash P(S) \implies (P(P(S)) \implies P(A))$.

By condition 3, $T\vdash P(S)\implies P(P(S))$ and thus $T\vdash P(S)\implies P(A)$.

By our initial assumption, this means that $T\vdash P(S)\implies A$.

But by the construction of $S$, then $T\vdash S$!

By condition 1 again, then $T\vdash P(S)$, and since we already had shown that $T\vdash P(S)\implies A$, we have that $T\vdash A$, which completes the proof.

We remark that $P$ can be any predicate satisfying the conditions, such as the verum predicate $x=x$.

M Yass
Dylan Hendrickson
Malcolm McCrimmon
Eric Bruylant
Jaime Sevilla Molina

You can prove [https://arbital.com/p/55w](https://arbital.com/p/55w) by assuming $\square A\to A$ and constructing a sentence $S$ such that $PA\vdash S\leftrightarrow (\square S \to A)$


Diagonal lemma

The **diagonal lemma** shows how to construct self-referential sentences in the language of arithmetic from formulas $\phi(x)$.

In its standard form it says that is $T$ is a theory extending [minimal arithmetic](https://arbital.com/p/) (so that it can talk about [recursive](https://arbital.com/p/) expressions with full generality) and $\phi(x)$ is a formula with one free variable $x$ then there exists a sentence $S$ such that $T\vdash S\leftrightarrow \phi(\ulcorner S\urcorner)$.

This can be further [generalized](https://arbital.com/p/) for cases with multiples formulas and variables.

The diagonal lemma is important because it allows the formalization of all kind of self-referential and apparently paradoxical sentences.

Take for example the liar's sentence affirming that "This sentence is false". Since [there is no truth predicate](https://arbital.com/p/), we will have to adapt it to our language to say "This sentence is not provable". Since there is a [predicate of arithmetic expressing provability](https://arbital.com/p/5gt) we can construct a formula $\neg \square_{PA} (x)$, which is true iff there is no proof in [$PA$](https://arbital.com/p/3ft) of the sentence [encoded](https://arbital.com/p/31z) by $x$.

By invoking the diagonal's lemma, we can see that there exists a sentence $G$ such that $PA\vdash G\leftrightarrow \neg \square_{PA} (\ulcorner G\urcorner)$, which reflects the spirit of our informal sentence. The [weak form of Gödel's first incompleteness theorem](https://arbital.com/p/) follows swiftly from there.

The equivalent in computation of the diagonal lemma is called [quining](https://arbital.com/p/322), and refers to computer programs which produce their own source code as part of their computation.

Indeed, the key insight of the diagonal lemma and quining is that you can have subexpressions in your program that when "expanded" are identical to the original expression that contains them.

Eric Rogstad
Eric Bruylant
Jaime Sevilla Molina

If F(x) is a one place formula of First Order Logic and $T$ is a theory extending [minimal arithmetic](https://arbital.com/p/) then there exists a sentence $S$ such that $T\vdash S\iff F(\ulcorner S \urcorner)$.


Gödel's first incompleteness theorem

The **first incompleteness theorem** is a result about the existence of sentences of arithmetic that cannot be proved or disproved, no matter what axioms we take as true.

What Gödel originally proved was that every [$\omega$-consistent](https://arbital.com/p/) and [axiomatizable](https://arbital.com/p/) extension of Peano Arithmetic is incomplete, but the result was later refined to weaken the requirement of $\omega$-consistency to simple [https://arbital.com/p/-5km](https://arbital.com/p/-5km) and the set of theorems that the extension had to prove to that of [minimal arithmetic](https://arbital.com/p/).

The heart of both proofs is the [https://arbital.com/p/-59c](https://arbital.com/p/-59c), which allows us express self-referential sentences in the language of arithmetic.

This put an end to the dream of building a [https://arbital.com/p/-complete](https://arbital.com/p/-complete) [https://arbital.com/p/-5hh](https://arbital.com/p/-5hh) that axiomatized all of mathematics, since as soon as one was expressive enough to talk about arithmetic, incompleteness would kick in.

##Interpretation from model theory
The first incompleteness theorem highlights the impossibility of defining the [natural numbers](https://arbital.com/p/45h) with the usual operations of [addition](https://arbital.com/p/) and [multiplication](https://arbital.com/p/) in [first order logic](https://arbital.com/p/).

We already knew from Lowenhëim-Skolem theorem that there would be models of $PA$ which are not isomorphic to the usual arithmetic, but the first incompleteness theorem implies that some of those models disagree in the truth value of some theorems of this language (those are the undecidable sentences).

Every [consistent](https://arbital.com/p/) and [axiomatizable](https://arbital.com/p/) extension of [minimal arithmetic](https://arbital.com/p/) is incomplete


Proof of Gödel's first incompleteness theorem

##Weak form
The weak Gödel's first incompleteness theorem states that every [$\omega$-consistent](https://arbital.com/p/) [axiomatizable](https://arbital.com/p/) extension of minimal arithmetic is incomplete.

Let $T$ extend [https://arbital.com/p/-minimal_arithmetic](https://arbital.com/p/-minimal_arithmetic), and let $Prv_{T}$ be the [standard provability predicate](https://arbital.com/p/5gt) of $T$.

Then we apply the [diagonal lemma](https://arbital.com/p/59c) to get $G$ such that $T\vdash G\iff \neg Prv_{T}(G)$.

We assert that the sentence $G$ is undecidable in $T$. We prove it by contradiction:

Suppose that $T\vdash G$. Then $Prv_ {T}(G)$ is correct, and as it is a $\exists$-rudimentary sentence then it is [provable in minimal arithmetic](https://arbital.com/p/every_true_e_rudimentary_sentence_is_provable_in_minimal_arithmetic), and thus in $T$. So we have that $T\vdash Prv_ {T}(G)$ and also by the construction of $G$ that $T\vdash \neg Prv_{T}(G)$, contradicting that $T$ is consistent.

Now, suppose that $T\vdash \neg G$. Then $T\vdash  Prv_{T}(G)$. But then as $T$ is consistent there cannot be a standard proof of $G$, so if $Prv_{T}(x)$ is of the form $\exists y Proof_{T}(x,y)$ then for no natural number $n$ it can be that $T\vdash Proof_ {T}(\ulcorner G\urcorner,n)$, so $T$ is $\omega$-inconsistent, in contradiction with the hypothesis.

##Strong form

> Every [consistent](https://arbital.com/p/5km) and [https://arbital.com/p/-axiomatizable](https://arbital.com/p/-axiomatizable) extension of [https://arbital.com/p/-minimal_arithmetic](https://arbital.com/p/-minimal_arithmetic) is [incomplete](https://arbital.com/p/complete).

This theorem follows as a consequence of the [undecidability of arithmetic](https://arbital.com/p/) combined with the lemma stating that [any complete axiomatizable theory is undecidable](https://arbital.com/p/)


Multiplication of rational numbers (Math 0)

We've seen how to [add](https://arbital.com/p/55m) and [subtract](https://arbital.com/p/56x) pairs of [rational numbers](https://arbital.com/p/4zq).
But the [natural numbers](https://arbital.com/p/45h) have another operation on them: multiplication.

Remember, a given rational number represents what we get when we cut an apple into pieces all of the same size, then take some number %%note:Possibly *more* than we actually made, and possibly negative!%% of the little pieces.
The **product** of $\frac{a}{m}$ and $\frac{b}{n}$ %%note: Recall that $\frac{a}{m}$ is "$a$ copies of the little-piece we get when we cut an apple into $m$ equal pieces.%% is what we call "$\frac{a}{m}$ **multiplied by** $\frac{b}{n}$", and it answers the question "What happens if we do the procedure that would make $\frac{b}{n}$, but instead of starting by cutting one apple into $n$ pieces, we started by cutting $\frac{a}{m}$ apples into $n$ pieces?".

We write the product of $\frac{a}{m}$ and $\frac{b}{n}$ as $\frac{a}{m} \times \frac{b}{n}$.

# Example

It's hopefully easy to see that $1 \times \frac{b}{n} = \frac{b}{n}$.
Indeed, the definition is "what do we get if we would make $\frac{b}{n}$, but instead of starting by cutting one apple, we started by cutting $1$ apple?"; but that's just the same!
It's like saying "What if, instead of putting bread around my sandwich filling, I tried putting bread?" - I haven't actually changed anything, and I'll still get the same old sandwich %%note:or $\frac{b}{n}$%% out at the end.

How about $2 \times \frac{3}{5}$? (Strictly speaking, I should probably be writing $\frac{2}{1}$ instead of $2$, but this way saves a bit of writing. $\frac{2}{1}$ means "two copies of the thing I get when I cut an apple into one piece"; but an apple cut into one piece is just that apple, so $\frac{2}{1}$ just means two apples.)
Well, that says "instead of cutting one apple, we cut two apples" into $\frac{3}{5}$-sized pieces.

From now on, my pictures of apples will get even worse: rather than being circles, they'll now be squares.
It just makes the diagrams easier to understand.

![Two times three-fifths](http://i.imgur.com/tW78Nys.png)

In the picture, we have two apples (squares) which I've drawn next to each other, separated by a dashed line.
Then I've taken $\frac{3}{5}$ of the whole shape (shaded in red): that is, to the group of two apples I have done the procedure that would create $\frac{3}{5}$ if it were done to one apple alone.

Notice, though, that this divides neatly into $\frac{3}{5}$ of the left-hand apple, and $\frac{3}{5}$ of the right-hand apple.
So the red-shaded area comes to $\frac{3}{5} + \frac{3}{5}$, which you already know how to calculate: it is $\frac{6}{5}$.

# General integer times fraction

Can you work out, from the case of $2 \times \frac{3}{5}$ above, what $m \times \frac{a}{n}$ is, where $m$ is an integer?

%hidden(Show solution):
It is $\frac{a \times m}{n}$.

Indeed, the procedure to get $\frac{a}{n}$ is: we split $1$ into $n$ equal pieces, and then take $a$ of them.
So the procedure to get $m \times \frac{a}{n}$ is: we split $m$ into $n$ equal pieces, and then take $a$ of them.

But each of the pieces we've just made by splitting $m$—that is, those demarcated by the longer solid lines in the $2 \times \frac{3}{5}$ diagram above—can be viewed as being $m$ copies of what we get by splitting $1$.
(In the diagram above, we have $2$ copies of that which we get by splitting $1$: namely the two copies indicated by the dashed line.)
So we can view the second procedure as: we split $1$ into $n$ equal pieces %%note:In the diagram above, there are $5$ such equal pieces, and right now we're looking only at one square, not at both squares joined together.%%, and then take $a$ of them %%note:In the diagram above, $a$ is $3$: this has given us the red shaded bit of one of the squares.%%, and then do this $m$ times. %%note: In the diagram above, $m$ is $2$: we're finally looking at the two squares joined together into a rectangle.%%

This produces $a \times m$ pieces, each of size $\frac{1}{n}$, and hence the rational number $\frac{a \times m}{n}$.
%

You should check that you get the right answer for a different example: $-5 \times \frac{2}{3}$.
%%hidden(Show solution):
This is "do the procedure that makes $\frac{2}{3}$, but instead of starting with $1$, start with $-5$".

So we take five anti-apples, and divide them into thirds (obtaining $15$ anti-chunks of size $\frac{1}{3}$ each, grouped as five groups of three); and then we take two chunks out of each group of three, obtaining $10$ anti-chunks of apple in total.

So $-5 \times \frac{2}{3} = \frac{-10}{3}$, in accordance with the rule of $n \times \frac{a}{n} = \frac{a \times m}{n}$.
%%

# General fraction times fraction

# Order doesn't matter

Notice that while it was fairly obvious that order doesn't matter during addition (that is, $\frac{a}{m} + \frac{b}{n} = \frac{b}{n} + \frac{a}{m}$), because it's simply "putting two things next to each other and counting up what you've got", it's not all that obvious that the product of two fractions should be independent of the order we multiplied in.
However, you should check, from the general expression above, that it actually *is* independent of the order.

Why is this?
Why should it be that "do the procedure that made $\frac{b}{n}$, but starting from $\frac{a}{m}$ instead of $1$" and "do the procedure that made $\frac{a}{m}$, but starting from $\frac{b}{n}$ instead of $1$" give the same answer?

Well, remember the diagram we had for $2 \times \frac{3}{5}$ (remembering that that is "do the procedure that would make $\frac{3}{5}$, but instead of doing it to $1$, we do it to $2$):

![Two times three-fifths](http://i.imgur.com/tW78Nys.png)

What would we get if we rotated this diagram by a quarter-turn?

![Two times three-fifths, rotated](https://imgur.com/C61M0f2.png)

But wait! The shaded bit is just what we get when we do the procedure that makes $2$ (namely "put two copies of the shape next to each other"), but instead of doing it on the single (upper-most) square, we do it to the version of the number $\frac{3}{5}$ that is represented by the shaded bit of the upper-most square! And that is exactly what we would do to get $\frac{3}{5} \times 2$.

In general, $\frac{a}{m} \times \frac{b}{n}$ is the same as $\frac{b}{n} \times \frac{a}{m}$, because the two just "come from the same diagram, rotated by a quarter-turn".
They are measuring the same amount of stuff, because the amount of stuff in a diagram doesn't change simply because we rotated it.

## Another example

We'll do $\frac{-5}{7} \times \frac{2}{3}$.

# Meditation: why the notation makes sense

At this point, a digression is in order.
We have already seen the notation $\frac{a}{n}$ for "take an apple; divide it into $n$ pieces, each $\frac{1}{n}$-sized; and then take $a$ of the chunks".
In the language of multiplication that we've now seen, that is "do what we would do to make $a$, but do it starting from a $\frac{1}{n}$-chunk instead of $1$".
That is, $\frac{a}{n}$ is just $\frac{1}{n} \times a$.

And we can do that in a different way: we can take $a$ apples, divide each into $n$ chunks, and then just draw one of the chunks from each apple.
In the language of multiplication, that is just "do what we would do to make a $\frac{1}{n}$-chunk, but do it to $a$ instead of $1$".
That is, $\frac{a}{n} = a \times \frac{1}{n}$.

Recalling that $a$ is just $\frac{a}{1}$, our notation $\frac{a}{n}$ is simply the same as $\frac{a}{1} \times \frac{1}{n}$, as an instance of the "instant rule" $\frac{a}{1} \times \frac{1}{n} = \frac{a \times 1}{1 \times n} = \frac{a}{n}$.

# Inverses: putting things in reverse

Remember that we had "anti-apples" as a way of making nothing ($0$) by adding to some quantity of apples.
In a similar vein, we can "invert" multiplication.

Whenever $a$ is not $0$, we can find a rational number $\frac{c}{d}$ such that $\frac{a}{b} \times \frac{c}{d} = 1$.
(Notice that we've got $1$ as our "base point" now, rather than the $0$ that addition had.)

Indeed, using the instant rule, we see that $\frac{a}{b} \times \frac{c}{d} = \frac{a \times c}{b \times d}$, so to make $1$ we want $a \times c$ to be the same as $b \times d$.

But we can do that: if we let $c = b$ and $d = a$, we get the right thing, namely $\frac{a \times b}{b \times a} = \frac{a \times b}{a \times b} = \frac{1}{1} = 1$.

So $\frac{b}{a}$ works as an inverse to $\frac{a}{b}$.
And this is why we needed $a$ not to be $0$: because $\frac{b}{a}$ isn't actually a rational number unless $a$ is nonzero.

## Intuition

We've seen how this definition follows from the instant rule.
Where does it *actually* come from, though?

Multiplication is making a number by "doing to some quantity what would usually be done to the quantity $1$".