Poset: Examples

The standard $\leq$ relation on integers, the $\subseteq$ relation on sets, and the $|$ (divisibility) relation on natural numbers are all examples of poset orders.

Integer Comparison
================

The set $\mathbb Z$ of integers, ordered by the standard "less than or equal to" operator $\leq$ forms a poset $\langle \mathbb Z, \leq \rangle$. This poset is somewhat boring however, because all pairs of elements are comparable; such posets are called chains or [totally ordered sets](https://arbital.com/p/540). Here is its Hasse diagram.

![Truncated Hasse diagram](http://i.imgur.com/STsyMfJ.png)
%%%comment:
dot source (doctored in GIMP)

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width)
  edge [= "none"](https://arbital.com/p/arrowhead)
  a [= "-3"](https://arbital.com/p/label)
  b [= "-2"](https://arbital.com/p/label)
  c [= "-1"](https://arbital.com/p/label)
  d [= "0"](https://arbital.com/p/label)
  e [= "1"](https://arbital.com/p/label)
  f [= "2"](https://arbital.com/p/label)
  g [= "3"](https://arbital.com/p/label)
  rankdir = BT;
  a -> b
  b -> c
  c -> d
  d -> e
  e -> f
  f -> g
}
%%%
Power sets
=========

For any set $X$, the power set of $X$ ordered by the set inclusion relation $\subseteq$ forms a poset  $\langle \mathcal{P}(X), \subseteq \rangle$. $\subseteq$ is clearly [reflexive](https://arbital.com/p/5dy), since any set is a subset of itself. For $A,B \in \mathcal{P}(X)$,  $A \subseteq B$ and $B \subseteq A$ combine to give $x \in A \Leftrightarrow x \in B$ which means $A = B$. Thus, $\subseteq$ is [antisymmetric](https://arbital.com/p/5lt). Finally, for $A, B, C \in \mathcal{P}(X)$, $A \subseteq B$ and $B \subseteq C$ give $x \in A \Rightarrow x \in B$ and $x \in B \Rightarrow x \in C$, and so the [transitivity](https://arbital.com/p/573) of $\subseteq$ follows from the transitivity of $\Rightarrow$.

Note that the strict subset relation $\subset$ is the strict ordering derived from the poset $\langle \mathcal{P}(X), \subseteq \rangle$.

Divisibility on the natural numbers
===========================

Let [$\mathbb N$](https://arbital.com/p/45h) be the set of natural numbers including zero, and let $|$ be the divides relation, where $a|b$ whenever there exists an integer $k$ such that $ak=b$. Then $\langle \mathbb{N}, | \rangle$ is a poset. $|$ is reflexive because, letting k=1, any natural number divides itself. To see that $|$ is anti-symmetric, suppose $a|b$ and $b|a$. Then there exist integers $k_1$ and $k_2$ such that $a = k_1b$ and $b = k_2a$. By substitution, we have $a = k_1k_2a$. Thus, if either $k$ is $0$, then both $a$ and $b$ must be $0$. Otherwise, both $k$'s must equal $1$ so that $a = k_1k_2a$ holds. Either way, $a = b$, and so $|$ is anti-symmetric. To see that $|$ is transitive, suppose that $a|b$ and $b|c$. This implies the existence of integers $k_1$ and $k_2$ such that $a = k_1b$ and $b = k_2c$. Since by substitution $a = k_1k_2c$, we have $a|c$.


Ackermann function

The Ackermann function works as follows:

One may have noticed that addition, multiplication, and exponentiation seem to increase in "power", that is, when pitted against each other, it is easier to produce enormous numbers with exponentiation than with multiplication, and so on.

The Ackermann function produces a hierarchy of such growth operations, and ascends one step higher in the hierarchy each time.

Addition is the first operator in the hierarchy (though if we wanted, we could define [https://arbital.com/p/-addition_as_repeated_succession](https://arbital.com/p/-addition_as_repeated_succession) and declare addition the second operator in the hierarchy). The next operator in the hierarchy is produced by iterating the previous element in the hierarchy. For instance, the next few operators in the hierarchy are multiplication, exponentiation, and tetration:

Multiplication is iterated addition: $A \cdot B = \underbrace{A + A + \ldots A}_{B \text{ copies of } A}$

Exponentiation is iterated multiplication: $A^B = \underbrace{A \times A \times \ldots A}_{B \text{ copies of } A}$.

($A ^ B$ is written $A \uparrow B$ in [https://arbital.com/p/knuth_up_arrow_notation](https://arbital.com/p/knuth_up_arrow_notation).)

Tetration is iterated exponentiation. $A \uparrow\uparrow B = \underbrace{A^{A^{\ldots^A}}}_{B \text{ copies of } A}$ times.

$\uparrow^n$ just means $n$ up arrows, so we can also write tetration as: $A \uparrow^2 B = \underbrace{A \uparrow^1 (A \uparrow^1 (\ldots A))}_{B \text{ copies of } A}$

Now the pattern can be noticed and generalized, following the rule: $A \uparrow^n B = \underbrace{A \uparrow^{n-1} (A \uparrow^{n-1} (\ldots A))}_{B \text{ copies of } A}$

And now we can define the Ackermann function as $A(n) = n \uparrow^n n$.

This definition is relatively small, but the functions grow at incredible rates. [Wait But Why](http://waitbutwhy.com/2014/11/1000000-grahams-number.html) has a good intuitive description of how incredibly fast tetration, pentation, and hexation grow, but by the time we get to $A(6)$, the Ackermann function already grows far more quickly than that. $A(1)=1$, $A(2)=4$, and $A(3)$ cannot be written in the universe, as it has enormously more digits than our universe has subatomic particles.

Interestingly enough, the Ackermann function grows faster than all primitive recursive functions. This is related to a rather deep connection between the consistency strength of a mathematical theory (which sorts of results they are capable of proving) and the slowest-growing function for which the theory cannot prove "this function is defined on all integers."

Alexei Andreev
Eric Rogstad
Chris Barnett
Alex Appel
Nate Soares
Eric Bruylant


Fractional digits

When $b$ and $x$ are [integers](https://arbital.com/p/48l), $\log_b(x)$ has a few good interpretations. It's roughly the [length of $x$](https://arbital.com/p/416) when written in [base $b$](https://arbital.com/p/number_bases). More precisely, it's the [fair price of an $x$-digit](https://arbital.com/p/427) in terms of [$b$-digits](https://arbital.com/p/4sj). But what about when $x$ or $b$ is not a whole number? $\log_{3.16}(5.62) \approx 1.5$; how are we supposed to interpret this fact?

We could appeal to the definition of the logarithm, observe that $3.16^{1.5} \approx 5.62,$ and call it a day. But what does that mean? That 3.16 multiplied by itself one-and-a-half times is roughly 5.62? That 5.62, written in base 3.16, is about 1.5 digits long? That a 5.62-digit is worth about 1.5 times as much as a 3.16-digit? What does it mean to multiply a number by itself "half a time"? How would you even write a number down in "base 3.16"? What would a 3.16-digit even look like?

Let's say that you want to [store a lot of data](https://arbital.com/p/storing_numbers), and [digit wheels](https://arbital.com/p/42d) cost \$1. At that price, an object that can store 3 different values costs about 48¢, whereas an object that stores 4 different values costs about 60¢. What storage medium, then, costs exactly 50¢? I can think of two:

1. You and I could buy a digit wheel, split the costs, and share it.
2. You and I could both pay 50¢ each for a digit wheel, and then toss a coin to see who gets to use it.

How could we split the digit wheel? The digit wheel can be put into any one of ten different states, labeled 0, 1, 2, 3, 4, 5, 6, 7, 8, and 9. Here's one way to use it to encode two different messages: I choose a number $a$ between 0 and 1, then you choose a number $b$ between 0 and 4. Then we put the digit wheel into the state $5a + b.$ Thus, if you choose 0, the wheel will either say 0 or 5 (depending on which number I choose). If you choose 3, the wheel will either say 3 or 8 (depending on which number I chose). Each number you want to record (0-4) has two representations, and I choose whether we use the high one or the low one. Thus, the digit wheel records two messages: 7 means (high, 2) and so on.

Of course, this is not a fair way to split the digit wheel — you get to store a [5-message](https://arbital.com/p/3v9) whereas I only get to store a 2-message.%%note:Thus we see that a [10-digit](https://arbital.com/p/4sj) is worth a 5-digit plus a 2-digit, that is, $\log_{10}(5) + \log_{10}(2) = 1$.%% Can we do something fairer?

Well, the methodology can be used to share arbitrary $n$-digits between two people. If $x$ and $y$ are whole numbers such that $x \cdot y \le n$ then an $n$-digit can be used to store both an $x$-message and a $y$-message at once: You choose the [https://arbital.com/p/-quotient](https://arbital.com/p/-quotient) of $n$ divided by $x$, I choose the [https://arbital.com/p/-remainder](https://arbital.com/p/-remainder).

__Exercise:__ How would you use an $18$-digit to store both a $3$-message and a $6$-message? How do you store (2, 4) using that encoding?
%%hidden(Answer):
Here's one way: Let $a$ be the value of the 3-message (between 0 and 2) and $b$ be the value of the 6-message (between 0 and 5). Record $6a+b.$ Using this encoding, (2, 4) is stored as the number 16.

You can verify for yourself that there is exactly one way to store each (3-message, 6-message) pair in an 18-message under this encoding.
%%

(From this we can see that, when $n = x \cdot y,$ the cost of a $n$-digit is exactly equal to the cost of an $x$-digit plus the cost of a $y$-digit. In other words, $n = x \cdot y$ then $\log_b(x) + \log_b(y) = \log_b(n),$ no matter what base $b$ we're using.)

The fair split of the 10-digit, then, is to let us both store a 3-message: 3 is the highest integer $x$ such that $x \cdot x < 10.$ This isn't maximally efficient, though; that encoding only uses 9 of the 10 possibilities that we paid for, and therefore it's not worth our money: Why buy what is effectively a 3-message for 50¢ when the fair price of 3-messages is 48¢? This might be worth knowing if the market isn't selling objects that carry 3-messages, and both you and I just need a tiny bit more storage, but in general it's not worth it.

However, it becomes more worthwhile the more 10-digits we split. You might expect that, when splitting three digit wheels, we'd each get one digit wheel to ourselves plus a 3-message each from the third wheel, giving us 30 different possible messages each. However, we can do better than that if we're clever. If you pick a number $a$ between 0 and 30 (for a total of 31 different possibilities), and I also pick a number $b$ between 0 and 30, then the number $31a + b$ can always be stored on 3 digit wheels (because $31 \cdot 30 + 30 = 960 \le 999$), so if we're splitting three digit wheels, we can actually use them to store two 31-messages. ([Why?](https://arbital.com/p/)) This still isn't maximally efficient (the values from 961 to 999 are wasted), but it's a little better. And we can keep increasing the efficiency if we keep going: Five digit wheels can hold two 316-messages; 7 digit wheels can hold two 3162-messages, and so on. (You can verify those numbers yourself.)

As you can see, when splitting an $n$-digit, the fair split occurs at the largest whole number $x$ such that $x \cdot x \le n$, in which case we both get to record one $x$-message. This is why it is 316, and not 500, that can most naturally be seen as "about 2.5 decimal digits long:" The number that is 2.5 decimal digits long is the largest number $x$ such that, given five digits, both you and I can use those 5 digits to store independent $x$-messages. If you store a 500-message, that only leaves enough space for me to store a 200-message. If you only store a 300-message, that leaves enough space for me to store a 333-message. $x=316$ is the point where we can both store an $x$-message, because 316 is the largest whole number such that $x^2 \le 100000.$

(From this we can deduce that $\log_b(316) \approx \frac{5\log_b(10)}{2}$, no matter what the base: A 316-digit is worth about half as much as five 10-digits put together.)

The more 10-digits you split, the more efficient it gets — if we split 7 digit wheels, each of us gets to record a 3162-message; if we split 9 wheels, we each get to record a 31622-message, and so on. But the split is never maximally efficient, because 10 is not a [https://arbital.com/p/-square_number](https://arbital.com/p/-square_number): An $n$-digit can be split perfectly in half if and only if there is an $x$ such that $x \cdot x = n,$ and this can't happen if $n$ is an odd power of 10. Thus, splitting an odd number of digit wheels never makes the final digit wheel be worth _exactly_ 50¢.

You know what storage medium is worth exactly 50¢? A 50% chance of a 10-digit.

Let's say we pay 50¢ each for a digit wheel, and then toss a coin to see who gets to use it. How much does this decrease your storage costs? Half the time, you get to [reduce the number you have left to store](https://arbital.com/p/) by a factor of 10; the other half the time, you reduce the number you have left to store by a factor of 1 (i.e., not at all). How much does that reduce the number you have left to store, [on average](https://arbital.com/p/4b5)?

On average, your number gets reduced by the [https://arbital.com/p/-geometric_mean](https://arbital.com/p/-geometric_mean) of 1 and 10. Geometric means "average out" multiplications: Let's say that you have a thing and you're going to multiply it by 2, 12 and 9. At the end of the day, your thing is going to be 216 times larger, and you're going to have multiplied it by three numbers. Geometric means ask how big each multiplication was "on average" — instead of multiplying your thing by three different numbers, can we find one number $y$ such that $y \cdot y \cdot y = 216,$ such that multiplying your thing by $y$ three times is the same as multiplying by 2, 12, and 9 separately? The answer is yes, and $y = \sqrt[https://arbital.com/p/3](https://arbital.com/p/3){2 \cdot 12 \cdot 9} = 6$, as you can verify. Thus, multiplying your thing by 6 is equivalent, [in expectation](https://arbital.com/p/4b5), to multiplying your thing by a value selected [uniformly at random](https://arbital.com/p/uniform_random) from the list [12, 9](https://arbital.com/p/2,). (For more on this, see [Geometric mean: Intuition](https://arbital.com/p/).)

Similarly, if we might reduce the size of the number we want to store by a factor of 1, and we might reduce it by a factor of 10, then "on average" we're reducing it by a factor of $\sqrt[https://arbital.com/p/2](https://arbital.com/p/2){1 \cdot 10} \approx 3.16.$ Multiplying by $\sqrt[https://arbital.com/p/2](https://arbital.com/p/2){10}$ twice is the same as multiplying by 1 once and 10 once, so it's equivalent [in expectation](https://arbital.com/p/4b5) to multiplying by values selected uniformly at random from the list [10](https://arbital.com/p/1,).

If _three_ people are sharing a 10-digit, then the amount of digits each one gets to use in expectation is $\sqrt[https://arbital.com/p/3](https://arbital.com/p/3){1 \cdot 1 \cdot 10} \approx 2.15.$ So that's what a 2.15 digit looks like — a 10-digit shared between three people. This can be generalized to any probability, and any $n$-digit for $1 < n \le 10$ can be interpreted as a digit wheel that you have some probability of getting to use ([How?](https://arbital.com/p/)).

There are a few different things we can learn from this interpretation. It allows us to give a physical interpretation to facts like $\log_{3.16}(5.62) \approx 1.5$, which says that a 5.62-digit is worth one and a half times as much as a 3.16 digit. What's a 3.16 digit? It's (approximately) a 10-digit shared between two people, i.e., a 10-digit that you have a 50% chance of getting to use yourself. What's a 5.62-digit? It's a 10-digit that you have a 75% chance of getting to use yourself. It's 1.5x as valuable, because 75% is 1.5x as much as 50%. The upgrade from a 3.16-digit to a 5.62-digit constitutes 1.5x increase in my chance of getting to use the digit wheel.

The other thing that this interpretation tells us is that the digit worth exactly half an $n$-digit is the $\sqrt{n}$-digit, both because (1) if you want to use an $n$-digit to store two $x$-messages for some $x$, you need $x \cdot x$ to be at most $n$; and (2) if you have a 50% chance of getting to use an $n$-digit and a 50% chance of [nothing](https://arbital.com/p/1digit), then that's equivalent [on average](https://arbital.com/p/4b5) to a certainty of a $\sqrt{n}$-digit.

In case you haven't noticed, we've been implicitly using the fact that doubling a digit squares the number of possibilities you have available to you: With one 10-digit you can store 10 possibilities, with two you can store $10^2 = 100.$ Because two $n$-digits [emulate](https://arbital.com/p/) an $n^2$-digit, it's no surprise that half an $n$-digit is a $\sqrt{n}$-digit.

You might be wondering how these square root signs weaseled into a discussion about logarithms. You also might be starting to suspect that roots, logarithms, and exponentials are secretly three sides of the same coin. If so, [you're right](https://arbital.com/p/exp_log_root), though that's a topic for another day.

Lest you think that this gives us a complete interpretation of logarithms, note that it still doesn't explain what's going on when the inputs to the logarithm function are between 0 and 1. You might think, "well, now that I know what a 3.16-digit is, I can use that intuition to figure out what a 0.16-digit is" — but it's not quite as easy as that. A 3.16-digit can be interpreted as probabilistic access to a 10-digit, and the same is true for $x$-digits whenever $x > 1.$ But no matter how many people share a 10-digit, it's never worth _less_ than a 1-digit ([which is worthless](https://arbital.com/p/1digit)); $\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){10} > 1$ no matter how large $n$ gets. To understand $x$-digits for $0 < x < 1,$ we need one final puzzle piece, covered in the next post.

Malcolm McCrimmon
Nate Soares
Eric Rogstad
Alexei Andreev


Shannon

The shannon (Sh) is a unit of [https://arbital.com/p/-3xd](https://arbital.com/p/-3xd). One shannon is the difference in [entropy](https://arbital.com/p/info_entropy) between a [https://arbital.com/p/-probability_distribution](https://arbital.com/p/-probability_distribution) on a [https://arbital.com/p/-binary_variable](https://arbital.com/p/-binary_variable) that assigns 50% to each value, and one that assigns 100% to one of the two values.


Natural number

A **natural number** is a number like 0, 1, 2, 3, 4, 5, 6, ... which can be used to represent the count of an object. The set of natural numbers is $\mathbb N.$ Not all sources include 0 in $\mathbb N.$

Natural numbers are perhaps the simplest type of number. They don't include [negative numbers](https://arbital.com/p/48l), [fractional numbers](https://arbital.com/p/4zq), [irrational numbers](https://arbital.com/p/4bc), [imaginary numbers](https://arbital.com/p/4zw), or any of those complexities.

Thanks to their simplicity, natural numbers are often the first mathematical concept taught to children. Natural numbers are equipped with a notion of [https://arbital.com/p/-addition](https://arbital.com/p/-addition) ($2 + 3 = 5$ and so on) and [https://arbital.com/p/-multiplication](https://arbital.com/p/-multiplication) ($2 \cdot 3 = 6$ and so on), these are among the first mathematical operations taught to children.

Despite their simplicity, the natural numbers are a ubiquitous and useful mathematical object. They're quite useful for counting things. They represent all the possible [cardinalities](https://arbital.com/p/4w5) of finite [sets](https://arbital.com/p/3jz). They're also a useful [data structure](https://arbital.com/p/data_structure), in that numbers can be used to [encode](https://arbital.com/p/numeric_encoding) all sorts of data. Almost all of modern mathematics can be built out of natural numbers.

Eric Rogstad
Patrick Stevens
Nate Soares
Martin Epstein
Eric Bruylant
Jaime Sevilla Molina
Joe Zeng

0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, ...

The natural numbers are the numbers we use to count things.


Log as the change in the cost of communicating

When interpreting logarithms as [a generalization of the notion of "length"](https://arbital.com/p/416) and as [digit exchange rates](https://arbital.com/p/427), in both cases, multiplying the input to the logarithm base 10 by a factor of 10 caused the output to go up by one. Multiplying a number by 10 makes it one digit longer. If a [https://arbital.com/p/-42d](https://arbital.com/p/-42d) is worth \$1, then a 1000-digit is worth exactly \$1 more than a 100-digit, because you can [build](https://arbital.com/p/emulating_digits) a 1000-digit out of a 100-digit and a 10-digit. Thus, by symmetry _dividing_ an input to the logarithm base 10 by 10 makes the output go down by one: If you divide a number by 10, it gets one digit shorter; and any $n$-digit is worth \$1 more than a $\frac{n}{10}$-digit, because you can build an $n$-digit out of a $\frac{n}{10}$-digit and a 10-digit.

This strongly implies that $\log_{10}(\frac{1}{10})$ should equal $-1$. If a 1000-digit costs \$3, and a 100-digit costs \$2, and a 10-digit costs \$1, and a [1-digit is worthless](https://arbital.com/p/1digit), then, extrapolating the pattern, a $\frac{1}{10}$ should cost $-\$1.$ But what does that mean? What sort of digit is worth negative money? Can we give this extrapolation a physical intuition?

Yes, we can, by thinking in terms of how difficult it is to communicate information. Let's say that you and I are in separate rooms, connected only by a conveyor belt, upon which I can place physical objects like coins, dice, and [digit wheels](https://arbital.com/p/42d) that you can read. Let's imagine also that a third party is going to show me the whodunit of a game of [Clue](https://arbital.com/p/https://en.wikipedia.org/wiki/Cluedo), and then let me put some objects on the conveyor belt, and then send those objects into your room, and then ask you for the information. If you can reproduce it successfully, then we both win a lot of money. However, I have to pay for every object that I put on the conveyor belt, using the [fair prices](https://arbital.com/p/427).

Consider how much I have to pay to tell you the result of a clue game. The "whodunit" in a clue game consists of three pieces of information:

1. The name of the murderer, which is one of: Miss Scarlett, Professor Plum, Mrs. Peacock, Reverend Green, Colonel Mustard, or Mrs. White.
2. The room in which the murder occurred, which is either the kitchen, the ballroom, the conservatory, the dining room, the cellar, the billiard room, the library, the lounge, the hall, or the study.
3. The murder weapon, which is either the candlestick, the dagger, the lead pipe, poison, the revolver, the rope, or the wrench.

Thus, a typical whodunit might look like "Professor Plum, in the conservatory, with the revolver." That sentence is 55 letters long, so one way for me to transmit the message would be to purchase fifty five 29-digits (capable of holding any one of 26 letters, or a space, or a comma, or a period), and send you that sentence directly. However, that might be a bit excessive, as there are in fact only $6 \cdot 10 \cdot 7 = 420$ different possibilities (six possible murderers, ten possible locations, seven possible weapons). As such, I only actually need to buy a 6-digit, a 10-digit, and a 7-digit. Equivalently, I could purchase a single 420-digit (if such things are on sale). We have to agree in advance what the digits mean — for example, "the 6-digit corresponds to the murderer, in the order listed above; the 10-digit corresponds to the room, in the order listed above; the 7-digit corresponds to the weapon, in the order listed above;" but assuming we do, I can get away with much less than fifty five 29-digits.

__Exercise:__ If the only storage devices on sale are coins, how many do I need to buy to communicate the whodunit?
%%hidden(Answer):
Nine. 8 coins only gets you 256 possibilities, and we need at least 420.
%%

__Exercise:__ If the only storage devices on sale are dice, how many do I need to buy?
%%hidden(Answer):
Four. $6^3 < 420 < 6^4.$
%%

__Exercise:__ If I have to choose between all coins or all dice, which should I choose, at the fair prices?
%%hidden(Answer):
The coins. Four dice cost as much as $\log_2(6) * 4 \approx 10.33$ coins, and we can do the job with nine coins instead.
%%

__Exercise:__ If I can mix coins, dice, and digit wheels, what's the cheapest way to communicate the whodunit?
%%hidden(Answer):
One coin and three dice let you send the message at a cost of only $\log_2(2) + 3\cdot \log_2(6) \approx 8.75$ coins.
%%

Now, consider what happens when the third party tells you "Actually, in order to win, you also have to communicate the name of my favorite Clue suspect, which is Colonel Mustard. I already told the person in the other room that you need to communicate two suspects, and that you'll communicate my favorite Clue suspect second. I didn't tell them who my favorite Clue suspect was, though."

Now, the space of possible messages has gone up by a factor of six: There are 420 possible whodunits, and each can be paired with one of six possible "favorite suspects," for a total of 2520 possible messages. How does this impact my cost of communicating with you? My cost goes up by 1 die ($= \log_2(6)$ coins $= \log_{10}(6)$ digit wheels). When the space of possibilities goes up by a factor of 6, my costs of communication (measured, say, in coins) go up by $\log_2(6).$

Now let's say that the third party comes back in the room and tells you "Actually, I gave the person in the other room a logic puzzle that told them which room the murder happened in; they solved it, and now they know that the murder happened in the conservatory."

This _reduces_ the space of possible messages I need to send, by a factor of 10. Now that both you and I know that the murder happened in the conservatory, I only need to transmit the murderer, the weapon, and the favorite suspect — one of 252 possibilities. The space of possibilities was cut into a tenth of its former size, and my cost of communicating dropped by 1 digit wheel ($= \log_6(10)$ dice $= \log_2(10)$ coins).

On this interpretation, logarithms are measuring how much it costs to transmit information, in terms of some "base" medium (such as coins, dice, or digit wheels). Every time the space of possibilities increases by a factor of $n$, my communication costs increase by $\log_2(n)$ coins. Every time the space of possibilities decreases by a factor of $n$, my communication costs _drop_ by $\log_2(n)$ coins.

This is the physical interpretation of logarithms that you can put your weight on: $\log_b(x)$ measures how much more or less costly it will be to send a message (in terms of $b$-digits) when the space of possible messages changes by a factor of $x$. Paired with a physical interpretation of [fractional digits](https://arbital.com/p/44l), it can explain most of the [basic properties of the logarithm](https://arbital.com/p/log_properties):

1. $\log_b(1) = 0,$ because increasing (or decreasing) the space of possible messages by a factor of 1 doesn't affect your communication costs at all.
2. $\log_b(b) = 1,$ because increasing the space of possible messages by a factor of $b$ will increase your communication costs by exactly one $b$-digit.
3. $\log_b\left(\frac{1}{b}\right) = -1,$ because decreasing the space of possible messages by a factor of $b$ saves you one $b$-digit worth of communication costs.
4. $\log_b(x\cdot y) = \log_b(x) + \log_b(y),$ because if $n = x \cdot y$ then one $n$-digit is exactly large enough to store one $x$-message and one $y$-message. Thus, when communicating, an $x\cdot y$-digit is worth the same amount as one $x$-digit plus one $y$-digit.
5. $\log_b(x^n) = n \cdot log_b(x),$ because $n$ $x$-digits can be used to [emulate](https://arbital.com/p/emulating_digits) one $x^n$-digit.

You might be thinking to yourself:

> Wait, what does it mean for the space of possible messages to go up or down by a factor of $x$? This isn't always clear. What if you're really good at guessing who people's favorite suspect is? For that matter, what if we haven't established a convention like "0 = Miss Scarlett; 1 = Professor Plum; ..."? If I see an observation, the amount by which it changes the space of possible messages is subjective; it depends on my beliefs and on the beliefs of the person I'm communicating with and on the conventions that we set up beforehand. How do you actually formalize this idea?

Those are great questions. Down that path lies [https://arbital.com/p/3qq](https://arbital.com/p/3qq), a field which measures communication costs using logarithms, and which lets us formalize (and quantize) ideas such as the amount of information carried by a message (to a given observer). See the [https://arbital.com/p/information_theory_tutorial](https://arbital.com/p/information_theory_tutorial) for more on this subject.

With regard to logarithms, the key idea here is an interpretation of what $\log_b(x)$ is "really doing." Given an input like "how many possible messages are there," such that your costs go up by 1 unit every time the input space increases by a factor of $b$, $\log_b(x)$ measures the change in cost when the input space increases by a factor of $x$. As we will see next, this idea generalizes beyond the domain of "set of possible messages vs cost of communicating," to _any_ scenario where some measure $\mu$ increases by $1$ every time some object scales by a factor of $b$, in which case $\log_b(x)$ measures the change in $\mu$ when the object scales by a factor of $x$. This is the defining characteristic of logarithms, and now that we have some solid physical interpretations of what it means, we're ready to start exploring logarithms in the abstract.


Lattice (Order Theory)

A **lattice** is a [https://arbital.com/p/-3rb](https://arbital.com/p/-3rb) that is closed under binary [joins and meets](https://arbital.com/p/3rc).

Let $L$ be a lattice. Then for all $p,q,r \in L$ the following properties are necessarily satisfied.

* [Associativity](https://arbital.com/p/3h4) of joins and meets: $(p \vee q) \vee r = p \vee (q \vee r)$, and $(p \wedge q) \wedge r = p \wedge (q \wedge r)$
* [Commutativity](https://arbital.com/p/3jb) of joins and meets: $p \vee q = q \vee p$ and $p \wedge q = q \wedge p$
* Idempotency of joins and and meets: $p \vee p = p$ and $p \wedge p = p$
* Absorption: $p \vee (p \wedge q) = p$ and $p \wedge (p \vee q) = p$

%%hidden(Proofs):

Lemma 1: Let $P$ be a poset, $S \subseteq P$, and $p \in P$. If both $\bigvee S$ and $(\bigvee S) \vee p$ exist then $\bigvee (S \cup \{p\})$ exists as well, and $(\bigvee S) \vee p = \bigvee (S \cup \{p\})$.

Proof: See the *Join fu* exercise in [https://arbital.com/p/4ll](https://arbital.com/p/4ll).

## Associativity
Let $L$ be a lattice and $p,q,r,s \in L$ such that $s = p \vee (q \vee r)$. We apply the above lemma, along with commutativity and closure of lattices under binary joins, to get
$$p \vee (q \vee r) = (q \vee r) \vee p = (\bigvee \{q, r\}) \vee p = \bigvee (\{q, r\} \cup \{p\}) =$$
$$\bigvee \{ q, r, p \} = \bigvee (\{p, q\} \cup \{r\}) = (\bigvee \{p, q\}) \vee r = (p \vee q) \vee r.$$

By duality, we also have the associativity of binary meets.

## Commutativity

Let $L$ be a lattice and $p,q \in L$. Then $p \vee q = \bigvee \{ p, q \} = q \vee p$. Binary joins are therefore commutative. By duality, binary meets are also commutative.

## Idempotency

Let $L$ be a lattice and $p \in L$. Then $p \vee p = \bigvee \{ p \} = p$. The property that for all $p \in L$, $p \vee p = p$ is called *idempotency*. By duality, we also have the idempotency of meets: for all $p \in L$, $p \wedge p = p$.

## Absorption

Since $p \wedge q$ is the greatest *lower bound* of $\{p,q\}$, $p \wedge q \leq p$. Because $p \leq p$ and $(p \wedge q) \leq p$, $p$ is an upper bound of $\{p, p \wedge q\}$, and so $p \vee (p \wedge q) \leq p$. On the other hand, $p \vee (p \wedge q)$ is the least *upper bound* of $\{p, p \wedge q\}$, and so $p \leq p \vee (p \wedge q)$. By anti-symmetry,  $p = p \vee (p \wedge q)$.

%%

Closure under finite joins and meets
--------------------------------------------------------------

Let $L$ be a lattice and $S = \{ s_1, ..., s_n \}$ be some finite subset of $L$. Then an inductive argument shows that $\bigvee S$ exists.

%%hidden(Proof):
Here again, we will need Lemma 1, stated in the proofs of the four lattice properties.

Our proof proceeds by induction on the cardinality of $S$.

The base case is $\bigvee \{ s_1 \} = s_1 \in L$. For the inductive step, we suppose that $\bigvee \{s_1, ..., s_i \}$ exists. Then, applying lemma 1, we have $\bigvee \{s_1, ..., s_{i+1} \} = \bigvee \{s_1, ..., s_i \} \vee s_{i+1}$. Applying our inductive hypothesis and closure under binary joins, we have $\bigvee \{s_1, ..., s_i \} \vee s_{i+1}$ exists. Lattices are therefore closed under all *finite* joins, not just binary ones. Dually, lattices are closed under all finite meets.
%%

Basic positive examples
--------------

Here are two Hasse diagrams of posets which are lattices.

![A diamond shaped lattice](http://i.imgur.com/OlQnU07.png)

%%%comment:

dot source:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width)
  edge [= "none"](https://arbital.com/p/arrowhead)
  a [= ""](https://arbital.com/p/label)
  b [= ""](https://arbital.com/p/label)
  c [= ""](https://arbital.com/p/label)
  d [= ""](https://arbital.com/p/label)

  rankdir = BT;
  a -> b
  a -> c
  b -> d
  c -> d
}
%%%

![A cube shaped lattice](http://i.imgur.com/L0x074n.png)

%%%comment:

dot source:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width)
  edge [= "none"](https://arbital.com/p/arrowhead)
  a [= ""](https://arbital.com/p/label)
  b [= ""](https://arbital.com/p/label)
  c [= ""](https://arbital.com/p/label)
  d [= ""](https://arbital.com/p/label)
  e [= ""](https://arbital.com/p/label)
  f [= ""](https://arbital.com/p/label)
  g [= ""](https://arbital.com/p/label)
  h [= ""](https://arbital.com/p/label)

  rankdir = BT;
  a -> b
  a -> c
  a -> d
  b -> e
  b -> f
  c -> e
  c -> g
  d -> f
  d -> g
  e -> h
  f -> h
  g -> h
}

%%%

Basic negative examples
------------------------------------

Here are two Hasse diagrams of posets which are *not* lattices.

![A simple non-lattice](http://i.imgur.com/DAeuYz0.png)

%%%comment:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width)
  edge [= "none"](https://arbital.com/p/arrowhead)
  a [= ""](https://arbital.com/p/label)
  b [= ""](https://arbital.com/p/label)
  c [= ""](https://arbital.com/p/label)
  d [= ""](https://arbital.com/p/label)
  e [= ""](https://arbital.com/p/label)
  f [= ""](https://arbital.com/p/label)
  g [= ""](https://arbital.com/p/label)
  h [= ""](https://arbital.com/p/label)

  rankdir = BT;
  a -> b
  a -> c
  a -> d
  b -> e
  b -> f
  c -> e
  c -> g
  d -> f
  d -> g
  e -> h
  f -> h
  g -> h
}

%%%

In the above diagram, the two bottom elements have no common lower bounds. Therefore they have no meet, and so the depicted poset is not a lattice. However, it should be easy to verify that this poset is closed under binary joins.

![Another simple non-lattice](http://i.imgur.com/5Vqk87u.png)

%%%comment:

dot source:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width)
  edge [= "none"](https://arbital.com/p/arrowhead)
  a [= ""](https://arbital.com/p/label)
  b [= ""](https://arbital.com/p/label)
  c [= ""](https://arbital.com/p/label)
  d [= ""](https://arbital.com/p/label)

  rankdir = BT;
  a -> b
  c -> d
}

%%%

The Hasse diagram of this poset has two connected components. No element from the left component will have a meet or a join with any element from the right component. The depicted poset is therefore *not* a lattice.

The connecting lemma
--------------------

The connecting lemma states that for any lattice $L$ and $p,q \in L$, $p \vee q = p \Leftrightarrow q \leq p$ and dually, $p \wedge q = p \Leftrightarrow q \geq p$. This simple but important lemma is so named because it establishes a connection between the lattice's join operator and its underlying poset order.

%%hidden(Proof):

We prove $p \vee q = p \Leftrightarrow q \leq p$; the other part follows from duality. If $p \vee q = p$, then $p$ is an upper bound of both $p$ and $q$, and so $q \leq p$. Going the other direction, suppose $q \leq p$. Since $p$ is and upper bound of itself by reflexivity, it then follows that $p$ is an upper bound of $\{p, q\}$. There cannot be a lesser upper bound of $\{p, q\}$ because it would not be an upper bound of $p$. Hence, $p \vee q = p$.

%%

Lattices as algebraic structures
-------------------------------------

It's also possible to formulate lattices as [algebraic structures](https://arbital.com/p/3gx) $\langle L, \vee, \wedge \rangle$ satisfying the associativity, commutativity, idempotency, and absorption laws described above. A poset $\langle L, \leq \rangle$ can then be defined such that for $p, q \in L$, $p \leq q$ whenever $p \vee q = q$. It can be shown that this poset is closed under binary meets and joins, and that these meets and joins are equal to the corresponding meets and joins of the algebraic lattice.

Additional material
-------------------------------

For more examples of lattices, see [https://arbital.com/p/574](https://arbital.com/p/574).
For some exercises involving the concepts introduced on this page, see [https://arbital.com/p/5ff](https://arbital.com/p/5ff).

Eric Rogstad
Kevin Clancy
Alexei Andreev

A **lattice** is a [https://arbital.com/p/-3rb](https://arbital.com/p/-3rb) that is closed under binary [joins and meets](https://arbital.com/p/3rc). It follows from this definition that lattices are closed under all *finite* joins and meets, and that for all lattices $L$ and elements $p, q \in L$, the following algebraic properties are satisfied:

* [Associativity](https://arbital.com/p/3h4) of joins and meets: $(p \vee q) \vee r = p \vee (q \vee r)$, and $(p \wedge q) \wedge r = p \wedge (q \wedge r)$
* [Commutativity](https://arbital.com/p/3jb) of joins and meets: $p \vee q = q \vee p$ and $p \wedge q = q \wedge p$
* Idempotency of joins and and meets: $p \vee p = p$ and $p \wedge p = p$
* Absorption: $p \vee (p \wedge q) = p$ and $p \wedge (p \vee q) = p$


Uncomputability

Imagine you were tasked with the mission of finding out which things can and cannot be calculated with a computer, given that you had infinite memory and infinite time.

Given the generality of computers, you might be tempted to say everything, but it turns out that this is not the case.

To begin tackling this problem, we need a formal notion of what it means to compute something. We are going to focus on computing **natural functions**; that is, [functions](https://arbital.com/p/3jy) which transform [natural numbers](https://arbital.com/p/45h) to natural numbers.

As natural numbers can be used to encode pretty much anything, this class of computations is far wider than one might initially think. For example, we can [https://arbital.com/p/-encode](https://arbital.com/p/-encode) finite sequences of arbitrary length of natural numbers in a single natural number, and we can encode words in sequences of numbers.

We are going to say that a computer program computes a certain natural function if on input $n$ it returns the result of applying such function to $n$, for every natural $n$. We note that programs do not have to halt on inputs; they may just run forever. In that case, we will say that the program is computing a partial function, which is not defined on some inputs.

%%%knows-requisite([https://arbital.com/p/2w0](https://arbital.com/p/2w0)):
Now, the first thing to notice is that the set of possible functions from $\mathbb{N}$ to $\mathbb{N}$ is not [https://arbital.com/p/-enumerable](https://arbital.com/p/-enumerable), while the set of possible programs, which are finite sequences of a finite quantity of symbols, is enumerable. Thus, there cannot be a one-to-one correspondence between functions and programs; there must necessarily exist a function which is not computed by any program.

So computers cannot compute everything!
%%%

----
##The diagonal function

We are going to construct a concrete example of a function which is not computable.

Imagine an infinite list of every possible Python program, associated with the infinite outputs each would produce if we feed it the sequence of natural numbers $1,2,3,...$.

Thus, we may end up with a table such as this:

program #1: 0->1, 1->X, 2->3,...

program #2:

etc...

Where an X means that the program does not halt on that input or does not return an output, and thus the function it represents is not defined there.

Now, we are going to construct an explicit function which is not computable using this table.

Let $DIAG$ be such that:
$$
DIAG(n) = \left\{
  \begin{array}{lr}
    1\ if\ M_n(n) = 0\\
    0\ if\ M_n(n)\mbox{ is undefined or defined and greater than 0}
  \end{array}
\right.
$$

The notation $M_n$ means *"the $n$th program in our enumeration of programs"*.
Let's see what we have done there. We are looking at the $n$th entry of every program and saying that this function on input $n$ outputs something different that the $n$th program. This technique is called [https://arbital.com/p/-diagonalization](https://arbital.com/p/-diagonalization), and it is ubiquitous in computability and logic.

The function $DIAG$, known as the [https://arbital.com/p/-diagonal_function](https://arbital.com/p/-diagonal_function), is guaranteed to disagree with every program on at least one input. Thus, there cannot be a program which computes $DIAG$!

----
##The halting function
The reader may at this point be however not satisfied with such an unnatural example of an uncomputable function. After all, who is going to want to compute such a weird thing? Do not dismay, for there is a much better example: the halting function.

Let $HALT$ be defined as:
$$
HALT(n,x) = \left\{
  \begin{array}{lr}
    1\ if\ M_n(x)\mbox{ halts}\\
    0\ if\ M_n(x)\mbox{ does not halt }
  \end{array}
\right.
$$

That is, the function $HALT$ decides whether a given program is going to halt on a particular input. Now that is interesting.

To cite one imaginative use of the halting function, one could for example code a program which on any input simply ignores the input and starts looking for a counterexample of the Collatz conjecture, halting [https://arbital.com/p/-46m](https://arbital.com/p/-46m) it finds one. Then we could use the halting function to see whether the conjecture is true or false.

Sadly, $HALT$ is not computable. We are going to give two proofs of this fact, one by **reduction** to the diagonal function and other by **diagonalization**.

###Proof by reduction
Proofs by **reduction** are quite common in computability. We start by supposing that the function we are dealing with, in this case $HALT$, is computable, and then we proceed to use this assumption to show that if that was the case we could compute another function we know is uncomputable.

Suppose we have a program, which we are going to represent by $HALT$, which computes the halting function.

Then we could write a program such as this one:

    DIAGONAL(n):
        if HALT(n,n):
            return 1 if M_n==0 else return 0
    else return 0

Which computes the diagonal function. As we know such a program cannot possibly exist, we conclude by *reductio ad absurdum* that $HALT$ is not computable.

###Exercise
Prove that $HALT$ is reducible to $DIAGONAL$; ie, that if $DIAGONAL$ was computable we could compute $HALT$.

%%hidden(Show solution):
Suppose we want to know whether the program $PROG$ halts on input $n$. Then we first code the auxiliary program:

    AUX(x):
        PROG(n)
        return 0

That is, $AUX$ is the program which on any input first runs $PROG$ on input $n$ and then outputs $0$ if $PROG$ ever halts.

Then $DIAG(\ulcorner AUX\urcorner)==1$ iff $PROG$ halts on input $n$.
%%

###Proof by diagonalization
Another nifty proof of uncomputability comes from diagonalization.

As before, let's suppose $HALT$ is computable. Then we could write a program such as this one:

    prog(n):
        if HALT(n,n)==1:
           loop forever
        else return 0

The program halts on input $n$ iff $M_n$ does not halt on input $n$.

Now, let $\ulcorner prog\urcorner$ represent the number in the list of the program which occupies the $prog$ program. Here comes the question: does $prog$ halt when its input is $\ulcorner prog \urcorner$?

Supposing either yes or no leads to a contradiction, so we conclude that such a program cannot possibly exist, and it follows from the contradiction that $HALT$ is not computable.

---
So now you are ready to start your journey through the realm of computability!

Suggested next steps include examining other uncomputable functions such as the [busy beaver sequence](https://arbital.com/p/), or proceeding to [complexity theory](https://arbital.com/p/).

Eric Rogstad
Eric Bruylant
Jaime Sevilla Molina
Alexei Andreev


Iff

Iff is a shorthand for "if and only if". Its logical symbol is $\leftrightarrow$.

"A iff B" ($A \leftrightarrow B$) is quite distinct from "if A then B" ($A \rightarrow B$). Consider the stipulation "If the dog barks, then it will soon bite". This would not obligate the dog to bark a warning before biting. The "if" relation isn't symmetrical. As such, the dog might sometimes bite spontaneously, with no barking at all.

If we wanted to ensure that biting is always forewarned by barking, we would instead stipulate "**Iff** dog barks, then it will soon bite". This is equivalent to

* "the dog barks *if and only if* it will soon bite"

* "If the dog barks then it will soon bite, and if the dog bites it will have barked beforehand".

* "The dog barks only when it will soon bite"

With "iff", the implication runs in both directions.


Proof by contradiction

A proof by contradiction (a.k.a. *reductio ad absurdum*, reduction to absurdity) is a [strategy](https://arbital.com/p/5xz) used by mathematicians to show that a mathematical statement is true by proving that the [https://arbital.com/p/-negation](https://arbital.com/p/-negation) of that statement leads to being able to prove that two opposite statements are simultaneously true (a [https://arbital.com/p/-contradiction](https://arbital.com/p/-contradiction)).

##Outline
The outline of the strategy is as follows:

1. Suppose that what you want to prove is false.
2. Derive a contradiction from it.
3. Conclude that the supposition is wrong.

##Examples
To illustrate the concept, we will do a simple, non rigorous reasoning. Imagine yourself in the next situation:

You are a defense lawyer. Your client is accused of stealing the cookie from the cookie jar. You want to prove her innocence. Lets say you have evidence that the jar is still sealed. Reason as follows:

1. Assume she stole the cookie from the cookie jar.
2. Then she would have had to open the jar.
3. The jar is still sealed.
4. For the jar to be sealed and for her to have opened it is a contradiction.
5. Hence the assumption in 1 is false (given the deductions below it are true).
6. Hence she did not steal the cookie from the cookie jar.

Now we will work through an actual mathematical example: we will show that $\sqrt 2$ is not [rational](https://arbital.com/p/4zq); that is, it cannot be expressed as the division of two [natural numbers](https://arbital.com/p/45h).

1. We suppose that $\sqrt 2$ is rational, which means that there exist $a,b\in\mathbb{N}$ such that $\sqrt 2 = \frac{a}{b}$. Without loss of generality, we can suppose that $a$ and $b$ [have no common divisors](https://arbital.com/p/coprime), since otherwise we can just divide both numbers by their greatest common divisor to obtain a pair of numbers which satisfy both properties.
2. If this was the case, $b\sqrt2=a$. by squaring both sides, we arrive to $2b^2=a^2$. But then $2$ divides $a$, so we can express $a$ as $2n$ for some $n\in\mathbb{N}$. Substituing in the previous expression, we arrive to $2b^2 = 4n^2\implies b^2 =2 n^2$. By the same reasoning, $2$ must divide $b$, but then $a$ and $b$ have a common divisor! We have a contradiction.
3. We conclude that our original assumption that $\sqrt 2$ is rational must be false, and thus $\sqrt 2$ is not rational.

##When to use it
Proof by contradiction is one of the most useful techniques one can use to prove anything.

In particular, if you get stuck while doing a proof, resorting to proof by contradiction is a great way to keep exploring a problem from a different perspective. Even if you do not get to solve the problem, you may get a useful insight about the problem when performing the procedure of proof by contradiction.

Also, trying to do proof by contradiction may result in a [counterexample](https://arbital.com/p/), which dissolves the problem in question.

Eric Rogstad
Eric Bruylant
Mark Chimes
Jaime Sevilla Molina
Joe Zeng


Derivative

The derivative of $y$ with respect to $x$ describes the [https://arbital.com/p/-rate](https://arbital.com/p/-rate) at which $y$ changes, given a change in $x$. In particular, we consider how tiny changes in one variable affect another variable. To take the derivative of a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy), we can draw a line that is [https://arbital.com/p/-tangent](https://arbital.com/p/-tangent) to a graph of the function. The slope of the tangent line is the value of the derivative at that point. The derivative of a function $f(x)$ is itself a function: it returns, for any $x$, the slope of the line that is tangent to $f(x)$ at the point $(x, f(x))$.

[https://arbital.com/p/toc:](https://arbital.com/p/toc:)

## Examples ##

 - The time-derivative of your car's mileage is your car's speed (because your car's speed is how quickly your car's mileage changes *over time*)

 - The time-derivative of your car's speed is your acceleration (because acceleration means how quickly your *speed* is changing over time)

 - The time-derivative of human population size is the birth rate minus the death rate (because if we take the birth rate minus the death rate, that tells how quickly the population is changing over time)

 - The time-derivative of wealth is income minus spending (because if we take income minus spending, that tells us how your wealth is changing over time)

 - The time-derivative of blood pumped through your heart is the flow rate through the aortic valve (rates in general describe how something changes over time)

 - The time-derivative of charge on a capacitor is the current flowing to it (currents in general are time-derivatives of how much total stuff has flowed)

 - The time-derivative of how good of a life you'll have lived is how happy you are right now (according to [hedonic utilitarians](https://foundational-research.org/hedonistic-vs-preference-utilitarianism/))

Okay, let's take another stab at this. Time-derivatives, or derivatives "with respect to time," describe how things change over time. We can take derivatives with respect to other things too.

 - The derivative of your car's mileage with respect to how much fuel you've burned is your miles per gallon (because your mpg describes how your mileage changes per unit of fuel you burn)

 - The derivative of the temperature of a pot of water with respect to how much heat you blast it with is called the heat capacity of water

 - The derivative of [potential energy](https://en.wikipedia.org/wiki/Potential_energy) with respect to altitude is gravitational force (a higher up object has some stored energy that it would release if it fell; a lower down object has less "potential energy." The change in potential energy as you change the altitude is why there's a force in the first place.)

There were two goals with all those examples, one explicit, and one covert. The explicit one was to give you a sense for what derivatives are. The covert one was to quietly suggest that you will never understand the the way the world works unless you understand derivatives. But hey, look at you! You kind of understand derivatives already! Let's get to the math now, shall we?

## Setting Up The Math ##

You just got your new car.

![](http://o.aolcdn.com/commerce/autodata/images/USC30TSC021B021001.jpg)
It's a [Tesla](http://waitbutwhy.com/2015/06/how-tesla-will-change-your-life.html) because you care about the environment almost as much as you care about looking awesome. Your mileage is sitting at 0. The world is your oyster. At time $t = 0$, you put your foot on the accelerator. For the next few seconds, your mileage will be $4.7 t^2$, where your mileage is in meters, and $t$ is in seconds since you pressed your foot on the accelerator. Now the first question is this: if that equation tells us how many meters we've traveled after how many seconds, how fast are we going at any given point in time?

The astute reader will have noticed that this was the first example of a time-derivative that we gave: the time-derivative of your car's mileage is your car's speed. Let's think about this *sans* math for a second. If we know where we are at any time, we should be able to figure out how fast we're going. There isn't any extra information we need. The only question is how. Well, we take the derivative of the mileage with respect to time to get our speed. In other words:

$$\frac{\mathrm{d}}{\mathrm{d} t} mileage = speed$$

That means the derivative with respect to $t$, where $t$ is the time in seconds. But we know what the mileage is, *in terms of $t$*. Our mileage is just $4.7 t^2$. So we can write:

$$\frac{\mathrm{d}}{\mathrm{d} t} 4.7 t^2 = speed$$

## Solving The Math ##

Sorry to leave you hanging for a sec, but we're going to start with something a little simpler.

![](http://i66.tinypic.com/1p87pl.png)
$$distance\ traveled = 2t$$

If this is the graph of how far someone has traveled after how many seconds, we can see that every second they go 2 more meters. In other words, they are traveling 2 meters per second, which you might notice is the slope of this line. In general, the derivative of a function is like the slope of the function when you graph it out.

This works fine if our function is something like $distance\ traveled = 2t$. What if our function isn't a line though. What if it's $distance\ traveled = t^2$?

![](http://i64.tinypic.com/htu0pv.png)

Things that aren't lines don't have slopes. So if this is a graph of our distance traveled over time, it's not as easy to see how fast we were going. But let's say we want to see how fast we were going at $t=1$. If we zoom in enough on that curve, it will start to flatten out into a straight line until we can't tell the difference. The slope of *that* line is what gives us our speed. The process of taking a curve like this one, and getting the "slope" at any given point is called "taking the derivative."

Let's take the derivative of $d = t^2$, where $d$ is the distance and $t$ is the time. (We'll take the derivative with respect to $t$). Prepare yourself, now take a look at the graph down there.

![](http://i67.tinypic.com/mcxloj.png)

We know how to find the slope of a line if we're given two points, so we're going to do that, and then slowly move the points together until they're on top of each other. The coordinates of the points are shown above, and we can calculate the slope pretty easily by doing $\frac{\Delta d}{\Delta t}$. This gives us a slope of 2.

Now let's say that our first point is at $(t,t^2)$, that our second point is $h$ units to the right, so it's coordinates are $((t+h),(t+h)^2)$. Now we have:
$$∆d=(t+h)^2-t^2$$
$$∆t=(t+h) - t$$
Algebradabra:
$$∆d=2ht + h^2$$
$$∆t=h$$
$$\frac{\Delta d}{\Delta t}=\frac{2ht + h^2}{h}=2t+h$$

Now as we make $h$ really small, the points get closer and closer together, and the slope of the line becomes $2t$. So when $t$ is $1$, the slope is $2$. And when $t$ is $5$, the slope is $10$.

We say the derivative of $t^2$ is $2t$. With similar logic, you can show that the derivative of $4.7t^2$ is $9.4t$. And *that* means that if you put your foot on the accelerator of your new Tesla at time $t=0$, your speed after $t$ seconds will be $9.4t$. After 1 second, you'll be traveling 9.4 meters per second. After 3 seconds, you'll be going 28.2 meters per second (or 64 mph).

## Concluding ##

That's what derivatives are. The Tesla case was just one example of actually finding the derivative of something. Obviously, if our distance traveled had been some totally different function of time, the derivative would have been different. We found that the derivative of $t^2$ is $2t$. Below is a list of other derivatives. You can imagine that if the function on the left was our distance traveled after a time $t$, the function on the right would be our speed at a time $t$. (All of the $c$'s and $n$'s are constants.)
$$\frac{\mathrm{d} }{\mathrm{d} t}c=0$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct=c$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^2=2ct$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^2=3ct^2$$
$$\frac{\mathrm{d} }{\mathrm{d} t}ct^n=nct^{n-1}$$
$$\frac{\mathrm{d} }{\mathrm{d} t}e^t=e^t$$
$$\frac{\mathrm{d} }{\mathrm{d} t}sin(t)=cos(t)$$
$$\frac{\mathrm{d} }{\mathrm{d} t}cos(t)=-sin(t)$$

If you're up for it, try to use the method we showed for solving derivatives to verify some of these. Good luck!

## See also

If you enjoyed this explanation, consider exploring some of [Arbital's](https://arbital.com/p/3d) other [featured content](https://arbital.com/p/6gg)!

Arbital is made by people like you, if you think you can explain a mathematical concept then consider [https://arbital.com/p/-4d6](https://arbital.com/p/-4d6)!

Eric Bruylant
Patrick Stevens
Michael Cohen
Alexei Andreev


The characteristic of the logarithm

Consider the interpretation of logarithms as [the cost of communicating a message](https://arbital.com/p/45q). Every time the number of possible messages to send doubles, your communication costs increase by the price of a coin, or whatever cheaper [https://arbital.com/p/-storage_medium](https://arbital.com/p/-storage_medium) you have that can communicate one of two messages. It doesn't matter whether the number of possible messages goes from 4 to 8 or whether it goes from 4096 to 8192; in both cases, your costs go up by the price of a coin. It is the factor by which the set grew (or shrank) that affects the cost; not the absolute number of messages added (or removed) from the space of possibilities. If the space of possible messages halves, your costs go down by one coin, regardless of how many possibilities there were before the halving.

Algebraically, writing $f$ for the function that measures your costs, $c(x \cdot 2) =$ $c(x) + c(2),$ and, in general, $c(x \cdot y) =$ $c(x) + c(y),$ where we can interpret $x$ as the number of possible messages before the increase, $y$ as the factor by which the possibilities increased, and $x \cdot y$ as the number of possibilities after the increase.

This is the key characteristic of the logarithm: It says that, when the input goes up by a factor of $y$, the quantity measured goes up by a fixed amount (that depends on $y$). When you see this pattern, you can bet that $c$ is a logarithm function. Thus, whenever something you care about goes up by a fixed amount every time something else doubles, you can measure the thing you care about by taking the logarithm of the growing thing. For example:

- Consider the problem of checking whether a date is contained in a gigantic [sorted list](https://arbital.com/p/sorted_list) of dates. You can do this by jumping to the middle of the list, seeing whether your date is earlier or later than the date in the middle, and thereby cutting the search space in half. Each time you do this, you cut the list of dates you're searching for in half, and so the total number of elements you need to look at goes up by one every time the size of the list doubles. Thus, the cost of searching an ordered list grows logarithmically in the size of the list. See also [https://arbital.com/p/binary_search](https://arbital.com/p/binary_search).
- Consider a colony of bacteria where each bacterium in the colony reproduces once per day. Thus, the size of the colony roughly doubles each day. If you care about how long this colony of bacteria has been growing, you can measure the days by taking the logarithm of the number of bacteria in the colony. The logarithm (base 2) counts how many times the colony has doubled (and the log base 3 counts how many times it has tripled, and so on).
- The length of a number in [https://arbital.com/p/4sl](https://arbital.com/p/4sl) grows more-or-less logarithmically in the magnitude of the number: When the magnitude of the number goes up by a factor of 10, the number of digits it takes to write the number down grows by 1. However, this analogy is not perfect: Sometimes, multiplying a number by two does not increase its length (consider the number 300), and sometimes, dividing a number by 10 does not decrease its length by one digit (consider the number 1). See also [Length isn't quite logarithmic](https://arbital.com/p/log_v_length).

Conversely, whenever you see a $\log_2$ in an equation, you can deduce that someone wants to measure some sort of thing by counting the number of doublings that another sort of thing has undergone. For example, let's say you see an equation where someone takes the $\log_2$ of a [https://arbital.com/p/-1rq](https://arbital.com/p/-1rq). What should you make of this? Well, you should conclude that there is some quantity that someone wants to measure which can be measured in terms of the number of doublings in that likelihood ratio. And indeed there is! It is known as [](https://arbital.com/p/bayesian_evidence), and the key idea is that the strength of evidence for a hypothesis $A$ over its negation $\lnot A$ can be measured in terms of $2 : 1$ updates in favor of $A$ over $\lnot A$. (For more on this idea, see [What is evidence?](https://arbital.com/p/)).

In fact, a given function $f$ such that $f(x \cdot y) = f(x) + f(y)$ is almost guaranteed to be a logarithm function — modulo a few technicalities.

[https://arbital.com/p/checkbox](https://arbital.com/p/checkbox)

[Conditional text depending what's next on the path.](https://arbital.com/p/fixme:)

Eric Rogstad
Adom Hartell
Nate Soares
Alexei Andreev

Any time you find an output that adds whenever the input multiplies, you're probably looking at a (roughly) logarithmic relationship. For example, imagine storing a number using [digit wheels](https://arbital.com/p/42d). Every time the number goes up by a factor of 10, you need one additional digit wheel: It takes 3 wheels to store the number 500; 4 to store the number 5000; 5 to store the number 50000; and so on. Thus, the relationship between the magnitude of a number and the number of digits it takes to write down is logarithmic. This pattern is the key characteristic of the logarithm, and whenever you see an output adding when the input multiplies, you can measure the output using logarithms.


Group homomorphism

summary(Technical): Formally, given two groups $(G, +)$ and $(H, *)$ (which hereafter we will abbreviate as $G$ and $H$ respectively), a group homomorphism from $G$ to $H$ is a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) $f$ from the underlying set $G$ to the underlying set $H$, such that $f(a) * f(b) = f(a+b)$ for all $a, b \in G$.

A group homomorphism is a function between [groups](https://arbital.com/p/3gd) which "respects the group structure".

#Definition

Formally, given two groups $(G, +)$ and $(H, *)$ (which hereafter we will abbreviate as $G$ and $H$ respectively), a group homomorphism from $G$ to $H$ is a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) $f$ from the underlying set $G$ to the underlying set $H$, such that $f(a) * f(b) = f(a+b)$ for all $a, b \in G$.

#Examples

 - For any group $G$, there is a group homomorphism $1_G: G \to G$, given by $1_G(g) = g$ for all $g \in G$. This homomorphism is always [bijective](https://arbital.com/p/499).
 - For any group $G$, there is a (unique) group homomorphism into the group $\{ e \}$ with one element and the only possible group operation $e * e = e$. This homomorphism is given by $g \mapsto e$ for all $g \in G$. This homomorphism is usually not [injective](https://arbital.com/p/4b7): it is injective if and only if $G$ is the group with one element. (Uniqueness is guaranteed because there is only one *function*, let alone group homomorphism, from any set $X$ to a set with one element.)
 - For any group $G$, there is a (unique) group homomorphism from the group with one element into $G$, given by $e \mapsto e_G$, the identity of $G$. This homomorphism is usually not [surjective](https://arbital.com/p/4bg): it is surjective if and only if $G$ is the group with one element. (Uniqueness is guaranteed this time by the property proved below that the identity gets mapped to the identity.)
 - For any group $(G, +)$, there is a bijective group homomorphism to another group $G^{\mathrm{op}}$ given by taking inverses: $g \mapsto g^{-1}$. The group $G^{\mathrm{op}}$ is defined to have underlying set equal to that of $G$, and group operation $g +_{\mathrm{op}} h := h + g$.
 - For any pair of groups $G, H$, there is a homomorphism between $G$ and $H$ given by $g \mapsto e_H$.
 - There is only one homomorphism between the group $C_2 = \{ e_{C_2}, g \}$ with two elements and the group $C_3 = \{e_{C_3}, h, h^2 \}$ with three elements; it is given by $e_{C_2} \mapsto e_{C_3}, g \mapsto e_{C_3}$. For example, the function $f: C_2 \to C_3$ given by $e_{C_2} \mapsto e_{C_3}, g \mapsto h$ is *not* a group homomorphism, because if it were, then $e_{C_3} = f(e_{C_2}) = f(gg) = f(g) f(g) = h h = h^2$, which is not true. (We have used that the identity gets mapped to the identity.)

# Properties

- The identity gets mapped to the identity. ([Proof.](https://arbital.com/p/49z))
- The inverse of the image is the image of the inverse. ([Proof.](https://arbital.com/p/4b1))
- The [image](https://arbital.com/p/3lh) of a group under a homomorphism is another group. ([Proof.](https://arbital.com/p/4b4))
- The composition of two homomorphisms is a homomorphism. ([Proof.](https://arbital.com/p/4b6))


Cyclic group

summary(technical):  A [group](https://arbital.com/p/3gd) $G$ is **cyclic** if it has a single [generator](https://arbital.com/p/generator_mathematics): there is one [element](https://arbital.com/p/element_mathematics) $g$ such that every element of the group is a [power](https://arbital.com/p/) of $g$.

# Definition

A cyclic group is a group $(G, +)$ (hereafter abbreviated as simply $G$) with a single generator, in the sense that there is some $g \in G$ such that for every $h \in G$, there is $n \in \mathbb{Z}$ such that $h = g^n$, where we have written $g^n$ for $g + g + \dots + g$ (with $n$ terms in the summand).
That is, "there is some element such that the group has nothing in it except powers of that element".

We may write $G = \langle g \rangle$ if $g$ is a generator of $G$.

# Examples

 - $(\mathbb{Z}, +) = \langle 1 \rangle = \langle -1 \rangle$
 - The group with two elements (say $\{ e, g \}$ with identity $e$ with the only possible group operation $g^2 = e$) is cyclic: it is generated by the non-identity element. Note that there is no requirement that the powers of $g$ be distinct: in this case, $g^2 = g^0 = e$.
 - The integers [modulo](https://arbital.com/p/modular_arithmetic) $n$ form a cyclic group under addition, for any $n$: it is generated by $1$ (or, indeed, by $n-1$).
 - The [symmetric groups](https://arbital.com/p/497) $S_n$ for $n > 2$ are *not* cyclic. This can be deduced from the fact that they are not [abelian](https://arbital.com/p/3h2) (see below).

# Properties

## Cyclic groups are [abelian](https://arbital.com/p/3h2)
Suppose $a, b \in G$, and let $g$ be a generator of $G$. Suppose $a = g^i, b = g^j$. Then $ab = g^i g^j = g^{i+j} = g^{j+i} = g^j g^i = ba$.

## Cyclic groups are [countable](https://arbital.com/p/2w0)
The elements of a cyclic group are nothing more nor less than $\{ g^0, g^1, g^{-1}, g^2, g^{-2}, \dots \}$, which is an enumeration of the group (possibly with repeats).

Eric Bruylant
Mark Chimes
Patrick Stevens

The cyclic [groups](https://arbital.com/p/3gd) are the simplest kind of group; they are the groups which can be made by simply "repeating a single element many times". For example, the rotations of a polygon.


Integer

An **integer** is a [https://arbital.com/p/-54y](https://arbital.com/p/-54y) that can be represented as either a [https://arbital.com/p/-45h](https://arbital.com/p/-45h) or its [https://arbital.com/p/-additive_inverse](https://arbital.com/p/-additive_inverse). -4, 0, and 1,003 are examples integers. 499.99 is not an integer. Integers are real numbers; they are [rational numbers](https://arbital.com/p/-4zq); and they are not [fractions](https://arbital.com/p/-fraction) or [decimals](https://arbital.com/p/-decimal).

## A Mathier Definition ##

Instead of describing the properties of an integer we'll describe the membership rules for the [https://arbital.com/p/-3jz](https://arbital.com/p/-3jz) $\mathbb{Z}$. After we're done, anything that's been allowed into $\mathbb{Z}$ counts as an integer.

Start by putting $0$ and $1$ into $\mathbb{Z}$. Now, pick an element of $\mathbb{Z}$, pick another element of $\mathbb{Z}$, and add them together (you can pick the same element twice). Is that number in $\mathbb{Z}$ yet? No?! Well let's put it in there fast. We can do the same thing as before except instead of adding, we subtract, and if the difference isn't in $\mathbb{Z}$ yet, we put it in there. Anything that could be let into $\mathbb{Z}$ with these procedures is an integer.

This is not an efficient algorithm for building out $\mathbb{Z}$, but it does show the primary motivation for having integers in the first place. Natural numbers (positive integers) are closed under addition, meaning that if you add any two elements in the set, the sum will be in the set, but natural numbers are not closed under subtraction. Integers are what you get when you expand natural numbers to make a set that is closed under subtraction as well.

## Formal construction

Given access to the set $\mathbb{N}$ of [natural numbers](https://arbital.com/p/45h), we may construct $\mathbb{Z}$ as follows.
Take the collection of all pairs $(a, b)$ of natural numbers, and take the [https://arbital.com/p/-quotient](https://arbital.com/p/-quotient) by the [https://arbital.com/p/-53y](https://arbital.com/p/-53y) $\sim$ such that $(a,b) \sim (c,d)$ if and only if $a+d = b+c$.
(The intuition is that the pair $(a,b)$ stands for the integer $a-b$, and we take the quotient so that any given integer has just one representative.)

Writing $[https://arbital.com/p/a,b](https://arbital.com/p/a,b)$ for the equivalence class of the pair $(a,b)$, we define the [https://arbital.com/p/-55j](https://arbital.com/p/-55j) structure as:

- $[https://arbital.com/p/a,b](https://arbital.com/p/a,b) + [https://arbital.com/p/c,d](https://arbital.com/p/c,d) = [https://arbital.com/p/a+c,b+d](https://arbital.com/p/a+c,b+d)$
- $[b](https://arbital.com/p/a,) \times [d](https://arbital.com/p/c,) = [bc+ad](https://arbital.com/p/ac+bd,)$
- $[https://arbital.com/p/a,b](https://arbital.com/p/a,b) \leq [https://arbital.com/p/c,d](https://arbital.com/p/c,d)$ if and only if $a+d \leq b+c$.

This does define the structure of a totally ordered ring ([proof](https://arbital.com/p/)).

Michael Cohen
Alexei Andreev
Dylan Hendrickson
Patrick Stevens
Nate Soares
Eric Bruylant
Joe Zeng


Square visualization of probabilities on two events

$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
$$

summary:
$$
\newcommand{\true}{\text{True}}
\newcommand{\false}{\text{False}}
\newcommand{\bP}{\mathbb{P}}
$$

We can represent a [probability distribution](https://arbital.com/p/joint_probability_distribution_on_event) $\bP(A,B)$ over  two [events](https://arbital.com/p/event_probability) $A$ and $B$ as a square:

We could also represent $\bP$ by [factoring](https://arbital.com/p/factoring_probability), so using $\bP(A,B) = \bP(A)\; \bP(B \mid A)$ we'd make this picture:

Say we have two [events](https://arbital.com/p/event_probability), $A$ and $B$, and a [probability distribution](https://arbital.com/p/joint_probability_distribution_on_event) $\bP$ over whether or not they happen. We can represent $\bP$ as a square:

So for example, the [probability](https://arbital.com/p/1rh) $\bP(A,B)$ of both $A$ and $B$ occurring is the ratio of \[area of the dark red region\](https://arbital.com/p/the) to \[area of the entire square\](https://arbital.com/p/the):

Visualizing probabilities in a square is neat because we can draw simple pictures that highlight interesting facts about our probability distribution.

Below are some pictures illustrating:

* [independent events](https://arbital.com/p/4cf) (What happens if the columns and the rows in our square *both* line up?)

* [marginal probabilities](https://arbital.com/p/marginal_probability) (If we're looking at a square of probabilities, where's the probability $\bP(A)$ of $A$ or the probability $\bP(\neg B)$?)

* [conditional probabilities](https://arbital.com/p/1rj) (Can we find in the square the probability $\bP(B \mid A)$ of $B$ if we condition on seeing $A$? What about the conditional probability $\bP(A \mid B)$?)

* [factoring a distribution](https://arbital.com/p/factoring_probability) (Can we always write $\bP$ as a square? Why do the columns line up but not the rows?)

* the process of computing [joint probabilities](https://arbital.com/p/1rh) from [factored probabilities](https://arbital.com/p/factoring_probability)

Independent events
===

Here's a picture of the joint distribution of [two independent events](https://arbital.com/p/4cf) $A$ and $B$:

Now the rows for $\bP(B)$ and  $\bP(\neg B)$ line up across the two columns. This is because $\bP(B \mid A) = \bP(B) = \bP(B \mid \neg A)$. When $A$ and $B$ are independent, updating on $A$ or $\neg A$ doesn't change the probability of $B$.

For more on this visualization of independent events, see the aptly named [https://arbital.com/p/4cl](https://arbital.com/p/4cl).

Marginal probabilities
===

We can see the [marginal probabilities](https://arbital.com/p/marginal_probability) of $A$ and $B$ by looking at some of the blocks in our square. For example, to find the probability $\bP(\neg A)$ that $A$ doesn't occur, we just need to add up all the blocks where $\neg A$ happens: $\bP(\neg A) = \bP(\neg A, B) + \bP(\neg A, \neg B)$.

Here's the probability $\bP(A)$ of $A$, and the probability $\bP(\neg A)$ of $\neg A$:

Here's the probability $\bP(\neg B)$ of $\neg B$:

In these pictures we're dividing by the area of the whole square. Since the probability of anything at all happening is 1, we could just leave it out, but it'll be helpful for comparison while we think about conditionals next.

Conditional probabilities
===

We can start with some probability $\bP(B)$, and then *assume* that $A$ is true to get a [conditional probability](https://arbital.com/p/1rj) $\bP(B \mid A)$ of $B$. Conditioning on $A$ being true is like restricting our whole attention to just the possible worlds where $A$ happens:

Then the conditional probability of $B$ given $A$ is the proportion of these $A$ worlds where $B$ also happens:

If instead we condition on $\neg A$, we get:

So our square visualization gives a nice way to see, at a glance, the conditional probabilities of $B$ given $A$ or given $\neg A$:

 We don't get such nice pictures for $\bP(A \mid B)$:

More on this next.

Factoring a distribution
===

Recall the square showing our joint distribution $\bP$:

Notice that in the above square, the reddish blocks for $\bP(A,B)$ and $\bP(A,\neg B)$ are the same width and form a column; and likewise the blueish blocks for $\bP(\neg A,B)$ and $\bP(\neg A,\neg B)$. This is because we chose to [factor](https://arbital.com/p/factoring_probability) our probability distribution starting with $A$:

$$\bP(A,B) = \bP(A) \bP( B \mid A)\ .$$

Let's use the [equivalence](https://arbital.com/p/event_variable_equivalence) between [events](https://arbital.com/p/event_probability) and [binary random variables](https://arbital.com/p/binary_variable), so if we say $\bP( B= \true \mid A= \false)$ we mean $\bP(B \mid \neg A)$. For any choice of truth values $t_A \in \{\true, \false\}$ and $t_B \in \{\true, \false\}$, we have

$$\bP(A = t_A,B= t_B) = \bP(A= t_A)\; \bP( B= t_B \mid A= t_A)\ .$$

The first factor $\bP(A = t_A)$ tells us how wide to make the red column $(\bP(A = \true))$ relative to the blue column $(\bP(A = \false))$. Then the second factor $\bP( B= t_B \mid A= t_A)$ tells us the proportions of dark $(B = \true)$ and light $(B = \false)$ within the column for $A = t_A$.

We could just as well have factored by $B$ first:

$$\bP(A = t_A,B= t_B) = \bP(B= t_B)\; \bP( A= t_A \mid B= t_b)\ .$$

Then we'd draw a picture like this:

By the way, earlier when we factored by $A$ first, we got simple pictures of the probabilities $\bP(B \mid A)$ for $B$ conditioned on $A$.  Now that we're factoring by $B$ first, we have simple pictures for the conditional probability $\bP(A \mid B)$:

and for the conditional probability $\bP(A \mid \neg B)$:

Computing joint probabilities from factored probabilities
===

Let's say we know the factored probabilities for $A$ and $B$, factoring by $A$. That is, we know $\bP(A = \true)$, and we also know $\bP(B = \true \mid A = \true)$ and $\bP(B = \true \mid A = \false)$. How can we recover the joint probability $\bP(A = t_A, B = t_B)$ that $A = t_A$ is the case and also $B = t_B$ is the case?

Since

$$\bP(B = \false \mid A = \true) = \frac{\bP(A = \true, B = \false)}{\bP(A = \true)}\ ,$$

we can multiply the prior $\bP(A)$ by the conditional $\bP(\neg B \mid A)$ to get the joint $\bP(A, \neg B)$:

$$\bP(A = \true)\; \bP(B = \false \mid A = \true) = \bP(A = \true, B = \false)\ .$$

If we do this at the same time for all the possible truth values $t_A$ and $t_B$, we get back the full joint distribution:


Symmetric group

The notion that group theory captures the idea of "symmetry" derives from the notion of the symmetric group, and the very important theorem due to Cayley that every group is a subgroup of a symmetric group.

# Definition

Let $X$ be a [https://arbital.com/p/-3jz](https://arbital.com/p/-3jz). A [bijection](https://arbital.com/p/499) $f: X \to X$ is a *permutation* of $X$.
Write $\mathrm{Sym}(X)$ for the set of permutations of the set $X$ (so its elements are functions).

Then $\mathrm{Sym}(X)$ is a group under the operation of composition of functions; it is the *symmetric group on $X$*.
(It is also written $\mathrm{Aut}(X)$, for the *automorphism group*.)

We write $S_n$ for $\mathrm{Sym}(\{ 1,2, \dots, n\})$, the *symmetric group on $n$ elements*.

# Elements of $S_n$

We can represent a permutation of $\{1,2,\dots, n\}$ in two different ways, each of which is useful in different situations.

## Double-row notation

Let $\sigma \in S_n$, so $\sigma$ is a function $\{1,2,\dots,n\} \to \{1,2,\dots,n\}$.
Then we write $$\begin{pmatrix}1 & 2 & \dots & n \\ \sigma(1) & \sigma(2) & \dots & \sigma(n) \\ \end{pmatrix}$$
for $\sigma$.
This has the advantage that it is immediately clear where every element goes, but the disadvantage that it is quite hard to see the properties of an element when it is written in double-row notation (for example, "$\sigma$ cycles round five elements" is hard to spot at a glance), and it is not very compact.

## Cycle notation

[Cycle notation](https://arbital.com/p/49f) is a different notation, which has the advantage that it is easy to determine an element's order and to get a general sense of what the element does.
Every element of $S_n$ [can be expressed in ](https://arbital.com/p/49k).

## Product of transpositions

It is a useful fact that every permutation in a (finite) symmetric group [may be expressed](https://arbital.com/p/4cp) as a product of [transpositions](https://arbital.com/p/4cn).

# Examples

- The group $S_1$ is the group of permutations of a one-point set. It contains the identity only, so $S_1$ is the trivial group.
- The group $S_2$ is isomorphic to the [https://arbital.com/p/-47y](https://arbital.com/p/-47y) of order $2$. It contains the identity map and the map which interchanges $1$ and $2$.

Those are the only two [abelian](https://arbital.com/p/3h2) symmetric groups.
Indeed, in cycle notation, $(123)$ and $(12)$ do not commute in $S_n$ for $n \geq 3$, because $(123)(12) = (13)$ while $(12)(123) = (23)$.

- The group $S_3$ contains the following six elements: the identity, $(12), (23), (13), (123), (132)$. It is isomorphic to the [https://arbital.com/p/-4cy](https://arbital.com/p/-4cy) $D_6$ on three vertices. ([Proof.](https://arbital.com/p/group_s3_isomorphic_to_d6))

# Why we care about the symmetric groups

A very important (and rather basic) result is [Cayley's Theorem](https://arbital.com/p/49b), which states the link between group theory and symmetry.

%%%knows-requisite([https://arbital.com/p/4bj](https://arbital.com/p/4bj)):
# Conjugacy classes of $S_n$

It is a useful fact that the conjugacy class of an element in $S_n$ is precisely the set of elements which share its [cycle type](https://arbital.com/p/4cg). ([Proof.](https://arbital.com/p/4bh))
We can therefore [list the conjugacy classes](https://arbital.com/p/4bk) of $S_5$ and their sizes.
%%%

# Relationship to the [https://arbital.com/p/-4hf](https://arbital.com/p/-4hf)

The [https://arbital.com/p/-4hf](https://arbital.com/p/-4hf) $A_n$ is defined as the collection of elements of $S_n$ which can be made by an even number of [transpositions](https://arbital.com/p/4cn). This does form a group ([proof](https://arbital.com/p/4hg)).

%%%knows-requisite([https://arbital.com/p/4h6](https://arbital.com/p/4h6)):
In fact $A_n$ is a [https://arbital.com/p/-4h6](https://arbital.com/p/-4h6) of $S_n$, obtained by taking the quotient by the [sign homomorphism](https://arbital.com/p/4hk).
%%%


Bijective function

A bijective function is a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) which has an inverse. Equivalently, it is both [injective](https://arbital.com/p/injective_function) and [surjective](https://arbital.com/p/surjective_function).

Eric Rogstad
Mark Chimes
Patrick Stevens


Cayley's Theorem on symmetric groups

Cayley's Theorem states that every group $G$ appears as a certain subgroup of the [https://arbital.com/p/-497](https://arbital.com/p/-497) $\mathrm{Sym}(G)$ on the [https://arbital.com/p/-3gz](https://arbital.com/p/-3gz) of $G$.

# Formal statement

Let $G$ be a group.
Then $G$ is [isomorphic](https://arbital.com/p/49x) to a subgroup of $\mathrm{Sym}(G)$.

# Proof

Consider the left regular [action](https://arbital.com/p/3t9) of $G$ on $G$: that is, the function $G \times G \to G$ given by $(g, h) \mapsto gh$.
This [induces a homomorphism](https://arbital.com/p/49c) $\Phi: G \to \mathrm{Sym}(G)$ given by [https://arbital.com/p/-currying](https://arbital.com/p/-currying): $g \mapsto (h \mapsto gh)$.

Now the following are equivalent:

- $g \in \mathrm{ker}(\Phi)$ the [kernel](https://arbital.com/p/49y) of $\Phi$
- $(h \mapsto gh)$ is the identity map
- $gh = h$ for all $h$
- $g$ is the identity of $G$

Therefore the kernel of the homomorphism is trivial, so it is injective.
It is therefore bijective onto its [image](https://arbital.com/p/3lh), and hence an isomorphism onto its image.

Since [the image of a group under a homomorphism is a subgroup of the codomain of the homomorphism](https://arbital.com/p/4b4), we have shown that $G$ is isomorphic to a subgroup of $\mathrm{Sym}(G)$.


Group action induces homomorphism to the symmetric group

Just as we can [curry](https://arbital.com/p/currying) functions, so we can "curry" [homomorphisms](https://arbital.com/p/47t) and [actions](https://arbital.com/p/3t9).

Given an action $\rho: G \times X \to X$ of group $G$ on set $X$, we can consider what happens if we fix the first argument to $\rho$. Writing $\rho(g)$ for the induced map $X \to X$ given by $x \mapsto \rho(g, x)$, we can see that $\rho(g)$ is a [bijection](https://arbital.com/p/499).

Indeed, we claim that $\rho(g^{-1})$ is an inverse map to $\rho(g)$.
Consider $\rho(g^{-1})(\rho(g)(x))$.
This is precisely $\rho(g^{-1})(\rho(g, x))$, which is precisely $\rho(g^{-1}, \rho(g, x))$.
By the definition of an action, this is just $\rho(g^{-1} g, x) = \rho(e, x) = x$, where $e$ is the group's identity.

We omit the proof that $\rho(g)(\rho(g^{-1})(x)) = x$, because it is nearly identical.

That is, we have proved that $\rho(g)$ is in $\mathrm{Sym}(X)$, where $\mathrm{Sym}$ is the [https://arbital.com/p/-497](https://arbital.com/p/-497); equivalently, we can view $\rho$ as mapping elements of $G$ into $\mathrm{Sym}(X)$, as well as our original definition of mapping elements of $G \times X$ into $X$.

# $\rho$ is a homomorphism in this new sense

It turns out that $\rho: G \to \mathrm{Sym}(X)$ is a homomorphism.
It suffices to show that $\rho(gh) = \rho(g) \rho(h)$, where recall that the operation in $\mathrm{Sym}(X)$ is composition of permutations.

But this is true: $\rho(gh)(x) = \rho(gh, x)$ by definition of $\rho(gh)$; that is $\rho(g, \rho(h, x))$ because $\rho$ is a group action; that is $\rho(g)(\rho(h, x))$ by definition of $\rho(g)$; and that is $\rho(g)(\rho(h)(x))$ by definition of $\rho(h)$ as required.


Cycle notation in symmetric groups

There is a convenient way to represent the elements of a [https://arbital.com/p/-497](https://arbital.com/p/-497) on a finite set.

# $k$-cycle

A $k$-cycle is a member of $S_n$ which moves $k$ elements to each other cyclically.
That is, letting $a_1, \dots, a_k$ be distinct in $\{1,2,\dots,n\}$, a $k$-cycle $\sigma$ is such that $\sigma(a_i) = a_{i+1}$ for $1 \leq i < k$, and $\sigma(a_k) = a_1$, and $\sigma(x) = x$ for any $x \not \in \{a_1, \dots, a_k \}$.

We have a much more compact notation for $\sigma$ in this case: we write $\sigma = (a_1 a_2 \dots a_k)$.
(If spacing is ambiguous, we put in commas: $\sigma = (a_1, a_2, \dots, a_k)$.)
Note that there are several ways to write this: $(a_1 a_2 \dots a_k) = (a_2 a_3 \dots a_k a_1)$, for example.
It is conventional to put the smallest $a_i$ at the start.

Note also that a cycle's inverse is extremely easy to find: the inverse of $(a_1 a_2 \dots a_k)$ is $(a_k a_{k-1} \dots a_1)$.

For example, the double-row notation $$\begin{pmatrix}1 & 2 & 3 \\ 2 & 3 & 1 \\ \end{pmatrix}$$
is written as $(123)$ or $(231)$ or $(312)$ in cycle notation.

However, it is unclear without context which symmetric group $(123)$ lies in: it could be $S_n$ for any $n \geq 3$.
Similarly, $(145)$ could be in $S_n$ for any $n \geq 5$.

# General elements, not just cycles

Not every element of $S_n$ is a cycle. For example, the following element of $S_4$ has [order](https://arbital.com/p/4cq) $2$ so could only be a $2$-cycle, but it moves all four elements:
$$\begin{pmatrix}1 & 2 & 3 & 4 \\ 2 & 1 & 4 & 3 \\ \end{pmatrix}$$

However, it may be written as the composition of the two cycles $(12)$ and $(34)$: it is the result of applying one and then the other.
Note that since the cycles are disjoint (having no elements in common), [it doesn't matter in which order we perform them](https://arbital.com/p/49g).
It is a very important fact that [every permutation may be written as the product of disjoint cycles](https://arbital.com/p/49k).
If $\sigma$ is a permutation obtained by first doing cycle $c_1 = (a_1 a_2 \dots a_k)$, then by doing cycle $c_2$, then cycle $c_3$, we write $\sigma = c_3 c_2 c_1$; this is by analogy with function composition, indicating that the first permutation to apply is on the rightmost end of the expression.
(Be aware that some authors differ on this.)

## Order of an element

Firstly, a cycle has [order](https://arbital.com/p/4cq) equal to its length.
Indeed, the cycle $(a_1 a_2 \dots a_k)$ has the effect of rotating $a_1 \mapsto a_2 \mapsto a_3 \dots \mapsto a_k \mapsto a_1$, and if we do this $k$ times we get back to where we started.
(And if we do it fewer times - say $i$ times - we can't get back to where we started: $a_1 \mapsto a_{i+1}$.)

Now, suppose we have an element in disjoint cycle notation: $(a_1 a_2 a_3)(a_4 a_5)$, say, where all the $a_i$ are different.
Then the order of this element is $3 \times 2 = 6$, because:

- $(a_1 a_2 a_3)$ and $(a_4 a_5)$ are disjoint and hence commute, so $[https://arbital.com/p/(](https://arbital.com/p/()^n = (a_1 a_2 a_3)^n (a_4 a_5)^n$
- $(a_1 a_2 a_3)^n (a_4 a_5)^n$ is the identity if and only if $(a_1 a_2 a_3)^n = (a_4 a_5)^n = e$ the identity, because otherwise (for instance, if $(a_1 a_2 a_3)^n$ is not the identity) it would move $a_1$.
- $(a_1 a_2 a_3)^n$ is the identity if and only if $n$ is divisible by $3$, since $(a_1 a_2 a_3)$'s order is $3$.
- $(a_4 a_5)^n$ is the identity if and only if $n$ is divisible by $2$.

This reasoning generalises: the order of an element in disjoint cycle notation is equal to the [https://arbital.com/p/-least_common_multiple](https://arbital.com/p/-least_common_multiple) of the lengths of the cycles.

# Examples

- The element $\sigma$ of $S_5$ given by first performing $(123)$ and then $(345)$ is $(345)(123) = (12453)$. Indeed, the first application takes $1$ to $2$ and the second application does not affect the resulting $2$, so $\sigma$ takes $1$ to $2$; the first application takes $2$ to $3$ and the second application takes the resulting $3$ to $4$, so $\sigma$ takes $2$ to $4$; the first application does not affect $4$ and the second application takes $4$ to $5$, so $\sigma$ takes $4$ to $5$; and so on.

This example suggests a general procedure for expressing a permutation which is already in cycle form, in *disjoint* cycle form. It turns out that [this can be done in an essentially unique way](https://arbital.com/p/49k).

## Cycle type

The [https://arbital.com/p/-4cg](https://arbital.com/p/-4cg) is given by taking the list of lengths of the cycles in the disjoint cycle form.


Disjoint cycles commute in symmetric groups

Consider two [cycles](https://arbital.com/p/49f) $(a_1 a_2 \dots a_k)$ and $(b_1 b_2 \dots b_m)$ in the [https://arbital.com/p/-497](https://arbital.com/p/-497) $S_n$, where all the $a_i, b_j$ are distinct.

Then it is the case that the following two elements of $S_n$ are equal:

- $\sigma$, which is obtained by first performing the permutation notated by $(a_1 a_2 \dots a_k)$ and then by performing the permutation notated by $(b_1 b_2 \dots b_m)$
- $\tau$, which is obtained by first performing the permutation notated by $(b_1 b_2 \dots b_m)$ and then by performing the permutation notated by $(a_1 a_2 \dots a_k)$

Indeed, $\sigma(a_i) = (b_1 b_2 \dots b_m)[https://arbital.com/p/(](https://arbital.com/p/() = (b_1 b_2 \dots b_m)(a_{i+1}) = a_{i+1}$ (taking $a_{k+1}$ to be $a_1$), while $\tau(a_i) = (a_1 a_2 \dots a_k)[https://arbital.com/p/(](https://arbital.com/p/() = (a_1 a_2 \dots a_k)(a_i) = a_{i+1}$, so they agree on elements of $(a_1 a_2 \dots a_k)$.
Similarly they agree on elements of $(b_1 b_2 \dots b_m)$; and they both do not move anything which is not an $a_i$ or a $b_j$.
Hence they are the same permutation: they act in the same way on all elements of $\{1,2,\dots, n\}$.

This reasoning generalises to more than two disjoint cycles, to show that disjoint cycles commute.

In a symmetric group, if we are applying a collection of permutations which are each disjoint cycles, we get the same result no matter the order in which we perform the cycles.


Disjoint cycle notation is unique


Pi

Pi, usually written $π$, is a number equal to the ratio of a circle's [https://arbital.com/p/-circumference](https://arbital.com/p/-circumference) to its [https://arbital.com/p/-diameter](https://arbital.com/p/-diameter). The value of $π$ is approximately $3.141593$.

![](http://rlv.zcache.ca/mathematical_definition_of_pi_ceramic_tiles-r78c5d7b1adfa46d7b24e2c8bc3bb3f65_agtk1_8byvr_324.jpg)

If the length of a curve seems like an ill-defined concept to you (maybe you only understand how lines could have lengths), consider bigger and bigger [regular polygons](https://arbital.com/p/-regular_polygon) that make better and better approximations of the circle. As the number of sides $N$ of the polygon goes to $∞$, the perimeter will approach a length of $π$ times the diameter.

One could also define $π$ to be the area of a circle divided the area of a square, whose edge is the radius of the circle.

## What Kind of Number It Is ##

It's not an [https://arbital.com/p/48l](https://arbital.com/p/48l).

![](https://sensemadehere.files.wordpress.com/2015/03/pi.png?w=584)

If the diameter here is 1, then the perimeter of the hexagon is 3, the perimeter of the square is 4, and the circumference of the circle is in between. There are no integers between 3 and 4.

It's [not rational](https://en.wikipedia.org/wiki/Proof_that_%CF%80_is_irrational), [nor is it algebraic](https://en.wikipedia.org/wiki/Lindemann%E2%80%93Weierstrass_theorem#Transcendence_of_e_and_.CF.80). It's transcendental.

Eric Rogstad
Eric Bruylant
Michael Cohen


Complexity theory

**Complexity theory** is the study of the efficiency of algorithms with respect to several metrics, usually time and memory usage. Complexity theorists aim to classify different problems into [classes of difficulty](https://arbital.com/p/complexity_class) and study the relations that hold between the classes.

When studying [computability](https://arbital.com/p/), we are concerned with the identification of that which is or not computable in an ideal sense, without worrying about time or memory limitations.

However, often in practice we have to be more pragmatic. A program which takes a [googol](https://arbital.com/p/42m) years to run is not going to see much use. If you need more [GbBytes](https://arbital.com/p/) to solve a computational problem that atoms exist in the universe, you may as well go ahead and declare the problem unsolvable for all practical purposes.

**Complexity theory** raises the standards of computability in drawing the boundary between that which you can do with a computer and that which you cannot. It concerns the study of the [asymptotic behavior](https://arbital.com/p/oh_notation) of programs when fed inputs of growing size, in terms of the resources they consume. The kind of resources with which complexity theorists work more often are the *time* a program takes to finish and the highest *memory usage* %%note:the memory usage is frequently called **space complexity**%% in any given point of the execution.

Complexity theory allows us to have a deeper understanding of what makes an algorithm efficient, which in turn allows us to develop better and faster algorithms. Surprisingly enough, it turns out that proving that some computational problems are hard to solve has incredibly important practical applications in [https://arbital.com/p/-cryptography](https://arbital.com/p/-cryptography).

----

The abstract framework in which we develop this theory are [Turing machines](https://arbital.com/p/) and [decision problems](https://arbital.com/p/). In this context, the [time complexity](https://arbital.com/p/) is associated with the number of steps a TM takes to halt and return an output, while the [space complexity](https://arbital.com/p/) corresponds to the length of the tape we would need for the TM to never fall off when moving left or right.

One may worry that the complexity measures are highly dependent on the choice of computational framework used. After all, if we allow our TM to skip two spaces per step each time it moves it is going to take potentially half the time to compute something. However, the asymptotic behavior of complexity is [surprisingly robust](https://arbital.com/p/robustness_of_tm), though there are [some caveats](https://arbital.com/p/failure_of_strong_CT).

The most interesting characterization of complexity comes in the form of [complexity classes](https://arbital.com/p/), which break down the family of [decision problems](https://arbital.com/p/) into sets of problems which can be solved with particular constraints.

Perhaps the most important complexity class is [$P$](https://arbital.com/p/), the class of decision problems which can be efficiently computed %%note:For example, checking whether a graph is connected or not%%. The second best known class is [$NP$](https://arbital.com/p/), the class of problems whose solutions can be easily checked %%note:An example is factoring a number: it is hard to factor $221$, but easy to multiply $13$ and $17$ and check that $13 \cdot 17 = 221$%% . It is an open problem whether those two classes are [one and the same](https://arbital.com/p/4bd); that is, that every problem whose solutions are easy to check is also easy to solve.

There are many more important complexity classes, and it can be daunting to contemplate the sheer variety with which complexity theory deals. Feel free to take a guided tour though the [complexity zoo](https://arbital.com/p/4b9) if you want an overview of some of the most relevant.

An important concept is that of a [reduction](https://arbital.com/p/). Some complexity classes have problems such that if you were able to solve them efficiently you could translate other problems in the class to this one and solve them efficiently. Those are called [complete problems of a complexity class](https://arbital.com/p/).

-----

This page is meant to be a starting point to learn complexity theory from an entry level. If there is any concept which feels mysterious to you, try exploring the greenlinks in their order of appearance. If you feel like the concepts presented are too basic, try a different lens.

Eric Bruylant
Adom Hartell
Eric Rogstad
Jaime Sevilla Molina

Study of the computational resources needed to solve a problem, usually time and memory


Group isomorphism

A group isomorphism is a [https://arbital.com/p/-47t](https://arbital.com/p/-47t) which is [bijective](https://arbital.com/p/499).
We say that two groups are *isomorphic* if there is an isomorphism between them.

It turns out that isomorphism is a much more useful concept than true equality of [groups](https://arbital.com/p/-3gd), and it captures the idea that "these two objects are the same group": the isomorphism shows us how to relabel the elements to see that they are indeed the same group.

For example, the trivial group is in some sense "the only group with one element", but it can be instantiated in many different ways: as $(\{ a \}, +_a)$, or $(\{ b \}, +_b)$, and so on (where $+_x$ is the [https://arbital.com/p/-3kb](https://arbital.com/p/-3kb) taking $(x, x)$ to $x$).
They all behave in exactly the same ways for the purpose of group theory, but they are not literally identical.
They are all isomorphic, though: the map $\{a \} \to \{ b \}$ given by $a \mapsto b$ is an isomorphism of the respective groups.

Two groups are isomorphic if and only if they have the same [Cayley table](https://arbital.com/p/cayley_table), possibly with rearrangement of rows/columns and with relabelling of elements.