Associativity: Intuition

Associative functions can be interpreted as families of functions that reduce lists down to a single output by combining adjacent elements in any order. Alternatively, associativity can be seen as a generalization of "listyness," which captures and generalizes the "it doesn't matter whether you added [b](https://arbital.com/p/a,) to c or a to [c](https://arbital.com/p/b,), the result is [b, c](https://arbital.com/p/a,) regardless" aspect of lists.

There are many different ways for a function to be associative, so it is difficult to provide a single litmus test for looking at a function and telling whether it associates (aside from just checking the associative axiom directly). However, a few heuristics can be used to make a good guess.

[https://arbital.com/p/toc:](https://arbital.com/p/toc:)

# Associative operators as natural functions on lists

The [generalized associative law](https://arbital.com/p/3ms) says that an [associative](https://arbital.com/p/3h4) [function](https://arbital.com/p/3jy) $f : X \times X \to X$ gives rise to a method for combining any non-empty list of $X$ elements into a single output, where the order in which adjacent elements are combined doesn't affect the result. We can flip that result around, and interpret associative operators as the pairwise versions of a certain class of "natural" functions for combining the elements of a list.

On this interpretation, we start by noting that some methods for reducing a list down to a single element can be broken down into pairwise combinations of adjacent elements, while others can't. For example, when we're trying to compute $3 + 4 + 5 + 6,$ we can pick any two adjacent elements and start by combining those using the binary version of $+$. But when we're trying to compute `adjacent_ones([0, 0, 1, 1, 0](https://arbital.com/p/0,))` to check whether the list has any two adjacent ones, we're going to run into trouble if we only look for adjacent ones in the pairs (0, 0), (0, 1), and (1, 0).

The lists that can be reduced by pairwise combination of adjacent pairs have a nice locality property; the result can be computed only by looking at adjacent elements (without worrying about the global structure). Locality is a common idiom in physics and mathematics, so we might start by asking what sort of functions on lists have this locality property. The answer is "any list that can be broken down into pairwise combinations of elements such that the order doesn't matter." If we formalize that notion, we get the result that any function on lists with this locality property corresponds (in a one-to-one fashion) to an associative operation. Thus, we can view associativity as the mathematical formalization of this nice "locality" property on lists.

Empirically, this locality property turns out to be quite useful for math, physics, and in computer programming, as evidenced by the commonality of associative operators. See, for example, [https://arbital.com/p/3g8](https://arbital.com/p/3g8), or the pages on [semigroups](https://arbital.com/p/algebraic_semigroup) and [monoids](https://arbital.com/p/3h3).

# Associativity as a generalization of "list"

The above interpretation gives primacy to lists, and interprets associative operators in terms of natural functions on lists. We can invert that argument by treating associativity as a _generalization_ of what it means for something to act "list-like."

A list $[b, c, d, \ldots](https://arbital.com/p/a,)$ is a set of elements that have been combined by some "combiner" function, where the order of the elements matters, but the order _in which they were combined_ does not matter. For example, if we combine $a$ with $b$ (forming $[b](https://arbital.com/p/a,)$) and then combine that with $c$, then we get the same list as if we combine $b$ and $c$ into $[c](https://arbital.com/p/b,)$ first, and then combine $a$ with that.

The very fact that we can unambiguously say "the list $[b, c](https://arbital.com/p/a,)$" without worrying about the order that the elements were combined in means that lists are built out of an associative "combination" operator. On this interpretation, associativity is capturing part of the essence of listyness, and associativity _in general_ generalizes this notion. For example, associative operators are allowed to be a little forgetful about what exact elements you combined (e.g., 3 + 4 = 2 + 5) so long as you retain the "it doesn't matter what order you combine the things in" property. In other words, we can view associativity as "part of what it means to be list-like."

(One particularly important property of lists — namely, that they can be empty — is not captured by associativity alone. Associative operators on sets that have an element that acts like an "empty list" are called "monoids." For more on the idea of generalizing the notion of "list", refer to the page on [monoids](https://arbital.com/p/3h3).)

# Associative mechanisms

The above two interpretations give an intuition for what it _means_ that a function is associative. This still leaves open the question of _how_ a function can be associative. Imagine $f : X \times X \to Y$ as a [physical mechanism](https://arbital.com/p/3mb) of wheels and gears. Someone says "$f$ is associative." What does that _mean,_ in terms of the function's physical mechanisms? What should we expect to see when we pop the function open, given the knowledge that it "is associative"?

Recall that associativity says that the two methods for combining two instantiations of the function yield the same output:

![Associative paths](http://i.imgur.com/Ezs1P8l.png)

Thus, the ultimate physical test of associativity is hooking up two instantiations of $f$ as in the left diagram, and then checking whether dragging the mechanisms of the lower-right instantiation above the mechanisms of the upper-left instantiation (thereby reconfiguring the system according to the diagram on the right) causes the behavior of the overall system to change. What happens when the right-hand-side instantiation is given access to the middle input first versus second? Does that affect the behavior at all? If not, $f$ is associative.

This is not always an easy property to check by looking at the mechanisms of $f$ alone, and sometimes functions that appear non-associative (at first glance) turn out to be associative by apparent coincidence. In other words, there are many different ways for a function to be associative, so it is difficult to give a single simple criterion for determining associativity by looking at the internals of the function. However, we can use a few heuristics that help one distinguish associative functions from non-associative ones.

## Heuristic: Can two copies of the function operate in parallel?

$f$ is associative if, when using two copies of $f$ to reduce three inputs to one output, then changing whether the right-hand copy gets access to the middle tape first vs second does not affect the output. One heuristic for checking whether this is the case is to check whether both copies of $f$ can make use of the middle input at the same time, without getting in each other's way. If so, $f$ is likely associative.

For example, consider an implementation of $+$ that gets piles of poker chips as input (where a pile of $n$ chips represents the number $n$) and computes the output by simply sweeping all the poker chips from its input belts onto its output belt. To  make a function that adds three piles of chips together, you could set up two two-pile adders in the configuration of the diagram on the left, but you could also have two two-tape sweepers operating on three tapes in parallel, such that they both sweep the middle tape. This parallelization wouldn't change the output, and thus $+$ is associative.

By contrast, consider a mechanism that takes wooden blocks as input, and glues them together, and nails silver-colored caps on either end of the glued block. For example, if you put in a red block on the left and a blue block on the right, you get a silver-red-blue-silver block in the output. You could set up two copies of these like the diagram on the left, but if you tired to parallelize them, you'd get into trouble — each mechanism would be trying to nail one of its caps into the place that the other mechanism was attempting to apply glue. And indeed, this mechanism is non-associative.

This heuristic is imperfect. Some mechanisms that seem difficult to parallelize are still associative. For example, consider the multiplier mechanism, which takes two poker piles as input and puts a copy of the left pile onto the output tape for every chip in the right pile. It would be difficult to parallelize two copies of this function: One would be trying to count the chips in the middle pile while the other was attempting to copy the chips in the middle pile, and the result might not be pretty. However, multiplication _is_ associative, because a pile of $x$-many copies of a ($y$ copies of $z$)-many poker chips has the same number of chips as a pile of ($x$ copies $y$)-many copies of $z$-many poker chips.

## Heuristic: Does the output interpretation match both input interpretations?

Another (vaguer) heuristic is to ask whether the output of the function should _actually_ be treated as the same sort of thing as the input to the function. For example, recall the `adjacent_ones` function from above, which checks a list for adjacent ones, and returns `1` if it finds some and `0` otherwise. The inputs to `adjacent_ones` are 0 and 1, and the output is 0 or 1, but the output interpretation doesn't quite match the input interpretation: Intuitively, the output is _actually_ intended to mean "yes there were adjacent ones" or "not here weren't adjacent ones", and so applying `adjacent_ones` to the output of `adjacent_ones` is possible but ill-advised. If there is a mismatch between the output interpretation and at least one of the input interpretations, then the function probably isn't associative.

For example, imagine a person who is playing a game that works as follows. The board has three positions: red, green, and blue. The player's objective is to complete as many clockwise red-green-blue cycles as possible, without ever backtracking in the counter-clockwise direction.

![3-cycle board](http://i.imgur.com/bxCnXUs.png)

 Each turn, the game offers them a choice of one of the three spaces, and they get to choose whether or not to travel to that square or stay where they are. Clearly, their preferences depend on where they currently are: If they're on "red", "green" is a good move and "blue" is a bad one; but if they're on "blue" then choosing "green" is ill-advised. We can consider a binary function $f$ which takes their current position on the left and the proposed position on the right, and returns the position that the player prefers. For example, $f(red,blue)=red,$ $f(red,green)=green,$ $f(blue,blue)=blue,$ $f(blue,green=blue).$ In this case, the interpretation of the left input is a "player position," the interpretation of the right input is an "offered move", and the interpretation of the output is the resulting "player position." The output interpretation mismatches one of the input interpretations, which implies that $f$ probably isn't associative, and indeed it is not: $f(f(red, green), blue))=blue,$ whereas $f(red, f(green, blue))=red.$ The former expression can be interpreted as "where the player would be if they started at red, and were then offered green, and were then offered blue." The latter expression doesn't have a great interpretation, because it's feeding the output of $f(green, blue)$ (a player position) in as an "offered move."

If the interpretation of the output (in this case, "player position") mismatches the interpretations of at least one of the inputs, then the function likely isn't associative. However, this heuristic is also imperfect: The most obvious interpretations of the inputs and outputs to the subtraction function are "they're all just numbers," and subtraction still fails to associate.

## Further discussion

There are many different ways for a function to be associative, so it is difficult to give a simple litmus test. The ultimate test is always to imagine using two copies of $f$ to combine three outputs into one, and check whether the result changes depending on whether the left-hand copy of $f$ gets to run first (in which case it gets to access the second input belt at the source) or second (in which case its right-hand input is the right-hand copy's output). For examples of functions that pass or fail this ultimate test, refer to the [examples page](https://arbital.com/p/3mt).


List

A list is an ordered collection of objects, such as `[1, 2, 3](https://arbital.com/p/0,)` or `["blue", 0, "shoe"](https://arbital.com/p/"red",)`. Lists are generally denoted by square brackets, with elements separated by commas. Lists allow repetition (unlike [sets](https://arbital.com/p/3jz)) and are ordered (unlike [bags](https://arbital.com/p/3jk)).

Sometimes lists are typed, in which case they contain only one type of element. For example, `[1, 2, 3](https://arbital.com/p/0,)` is a well-typed list of numbers, whereas the list `["blue", 0, "shoe"](https://arbital.com/p/"red",)` is not well-typed (because it contains both numbers and [strings](https://arbital.com/p/3jr)). The set of all lists of type $X$ is usually denoted $[https://arbital.com/p/X](https://arbital.com/p/X)$ or $X^{\le \omega}$, where $\omega$ is the smallest infinite [limit ordinal](https://arbital.com/p/limit_ordinal). The set of all finite lists of type $X$ is usually denoted $X^{< \omega}$, and the set of all infinite lists of type $X$ is usually denoted $X^{\omega}$.


Codomain (of a function)

The codomain $\operatorname{cod}(f)$ of a [function](https://arbital.com/p/3jy) $f : X \to Y$ is $Y$, the set of possible outputs for the function. For example, the codomain of [concat](https://arbital.com/p/3jv) is the set of all [strings](https://arbital.com/p/3jr), and the codomain of the function $+$ is the set of all numbers.

Visualizing a function as a map that takes every point in an input set to one point in an output set, the codomain is the output set (pictured on the right in blue in the image below).

![Domain, Codomain, and Image](http://i.imgur.com/ZZgVHaI.png)

The codomain of a function is not to be confused with the [image](https://arbital.com/p/3lh) of a function, which is the set of points in $Y$ that can actually be reached by following $f$, and which may not include the whole set $Y$. For example, consider all the functions that take a [real number](https://arbital.com/p/real_number) as input and produce another real number. Many of those functions cannot be made to produce every possible real number: For example, the function $\operatorname{square} : \mathbb R \to \mathbb R$ only produces non-negative numbers. For more on the distinction, see the page on [codomain vs image](https://arbital.com/p/3lv).

[Add a lens talking about how codomains are arbitrary but often natural/useful. Use example like how we can consider $+$ to have codomain $\mathbb N$, $\mathbb Z$, etc., and examples like Ackerman where the codomain "nat" makes by far the most sense ](https://arbital.com/p/fixme:)


Image (of a function)

The image $\operatorname{im}(f)$ of a [function](https://arbital.com/p/3jy) $f : X \to Y$ is the set of all possible outputs of $f$, which is a subset of $Y$. Using [set builder notation](https://arbital.com/p/3lj), $\operatorname{im}(f) = \{f(x) \mid x \in X\}.$

Visualizing a function as a map that takes every point in an input set to one point in an output set, the image is the set of all places where $f$-arrows land (pictured as the yellow subset of $Y$ in the image below).

![Domain, Codomain, and Image](http://i.imgur.com/ZZgVHaI.png)

The image of a function is not to be confused with the [codomain](https://arbital.com/p/3lg), which is the _type_ of output that the function produces. For example, consider the [Ackermann function](https://arbital.com/p/43x), which is a very fast-growing (and difficult to compute) function. When someone asks what sort of thing the Ackermann function produces, the natural answer is not "something from a sparse and hard-to-calculate set of numbers that I can't tell you off the top of my head"; the natural answer is "it outputs a number." In this case, the codomain is "number", while the image is the sparse and hard-to-calculate subset of numbers. For more on this distinction, see the page on [codomain vs image](https://arbital.com/p/3lv).


Set builder notation

$\{ 2n \mid n \in \mathbb N \}$ denotes the set of all even numbers, using set builder notation. Set builder notation involves an expression on the left and a series of constraints on the right, separated by a pipe and placed between curly braces. The expression on the left makes use of variables that are introduced and constrained on the right. The result denotes the set of all possible values on the left-hand side that obey the constraints on the right-hand side. For example, $\{ (x, y) \mid x \in \mathbb R, y \in \mathbb R, x \cdot y = 1 \}$ is the set of all pairs of real numbers whose product is 1.


Range (of a function)

The "range" of a function is an ambiguous term that is generally used to refer to either the function's [codomain](https://arbital.com/p/3lg) or [image](https://arbital.com/p/3lh).


Codomain vs image

A function $f : X \to Y$ with [domain](https://arbital.com/p/3js) $X$ and [codomain](https://arbital.com/p/3lg) $Y$ cannot necessarily produce all values in $Y$. For example, consider the function that takes a [real number](https://arbital.com/p/real_number) and squares it. We could say that this function has codomain $\mathbb R$, because it produces a real number, even though its image is the set of non-negative real numbers.

![Domain, Codomain, and Image](http://i.imgur.com/EHBsE1C.png)

If $f$ maps $X$ to a subset $I$ of $Y$, why not just say that its codomain is $I$ and do away with the distinction between codomain and image? There are at least two cases where the distinction is useful.

1. It is useful to distinguish codomain from image in cases where the _type of thing_ that the function produces is easy to specify, but the _specific values_ are hard to calculate. Consider, for instance, the [Ackermann function](https://arbital.com/p/43x). It definitely produces numbers (i.e. its codomain is $\mathbb N$), but it's not easy to figure out what numbers (exactly) it produces. When talking about what sort of outputs to expect from the Ackermann function, the natural answer is "a number," rather than "either 1 through 7, 9, 11, 13, 29, 61, 125, 65533, $2^{65536} − 3$, or some other ungodly large numbers that I lack the resources to calculate." For this purpose, the codomain can be thought of as a natural/simple boundary drawn around a complex image.

2. Cases where we're considering a set of all functions that map into a certain set. For example, if we're discussing all possible functions from the [natural numbers](https://arbital.com/p/natural_number) to the set $\{0, 1\},$ we want to be able to talk about "functions with codomain $\{0, 1\}$" even if some of those functions are things like "always return 0" (which have an image smaller than $\{0, 1\}$). For this purpose, the codomain can be thought of as a context-dependent tool that's useful when considering (or making use of) a bunch of different functions that are all producing objects of the same type.

Eric Rogstad
Eric Bruylant
Nate Soares
Patrick Stevens

It is useful to distinguish codomain from image both (a) when the type of thing that the function produces is simple to describe, but the specific objects produced are complicated and hard to identify; and (b) when talking about (or making use of) a collection of functions from a set $X$ to a set $Y$, many of which have an image which is a subset of $Y$.


Function: Physical metaphor

Many functions can be visualized as physical mechanisms of wheels and gears, that take their inputs on conveyor belts, manipulate them (using mechanical sensors and tools), and produce an output which is placed on an outgoing conveyor belt.

![A binary function](http://i.imgur.com/lKxAXcK.png)

For example, we can visualize the function $+$ as a function that takes two stacks of poker chips in as input (one on the left conveyor belt and one on the right conveyor belt) and produces its output by dumping both stacks onto the output belt. We can visualize the function $\times$ as a function that also gets a stack of poker chips on each input belt, and which puts a copy of the left stack onto the output tape for each poker chip in the right stack.

All [computable](https://arbital.com/p/computable_function) functions can (in principle) be implemented by a physical system of wheels and gears (or circuits and wires, etc.), and it is conjectured that the computable functions are the _only_ functions that can be implemented by physical systems. (This conjecture is known as the [Church-Turing thesis](https://arbital.com/p/church_turing_thesis).)


Generalized associative law

Given an [associative](https://arbital.com/p/3h4) [operator](https://arbital.com/p/3h7) $\cdot$ and a [list](https://arbital.com/p/3kd) $[b, c, \ldots](https://arbital.com/p/a,)$ of parameters, all ways of reducing the list to a single element by combining adjacent pairs yield the same result. In other words, while associativity only talks about the behavior of $f$ when combining three elements, this is sufficient to ensure that $f$ has similar good behavior when combining arbitrarily many elements

For example, consider the list $[b, c, d, e](https://arbital.com/p/a,),$ and abbreviate $a \cdot b$ as $ab.$ $((ab)c)(de)$ then denotes the output created by first combining $a$ and $b$, and then combining that with $c$, and then combining $d$ and $e$, and then putting the two results together. $a(b(c(de))$ denotes the result of combining $d$ and $e$ first, and then combining $c$ with that, and then combining $b$ with that, and then combining $a$ with that. The generalized associative law says that all possible orders of reducing the list (by combining adjacent elements) yield the same result, which means we are justified in dropping all parenthesis (and just writing $abcde$ for the result of reducing the list $[b, c, d, e](https://arbital.com/p/a,)$ using $\cdot$), because the order of application does not matter.

Visually, given an associative [function](https://arbital.com/p/3jy) $f$, there are five ways to use three copies of $f$ to combine four inputs into one output:

![Five ways of hooking up three instantiations of f](http://i.imgur.com/OvLsjqr.png)

The generalized associative law says that all five yield the same result. Because all five combinations are equivalent, there is only one function $f_4$ that uses three copies of $f$ to combine four inputs into one output. Similarly, there is only one function $f_5$ that combines five elements into one using four applications of $f,$ and so on.

# Proof sketch

Associativity of $\cdot$ lets us shift parenthesis to the left or right when manipulating expressions that use $\cdot$ multiple times in a row, because associativity says that $(x\cdot y) \cdot z = x \cdot (y \cdot z).$ Abbreviating $x \cdot y$ as $xy,$ we can use this rule to prove the generalized associative law.

Let a "reduction path" of a list $[b, c, d](https://arbital.com/p/a,)$ be an expression of the form $a(b(cd)),$ where the parenthesis tell us explicitly how to use applications of $\cdot$ on adjacent elements to combine the elements of the list into a single output. To prove the generalized associative law for lists of length 4, observe that any reduction path can be turned into any other, by repeatedly shifting the parenthesis left or right. For example:

$a(b(cd))=a((bc)d)=(a(bc))d=((ab)c)d=(ab)(cd).$

In this case, the expression is transformed by shifting the inner parentheses left, then shifting the outer parentheses left, then shifting the inner parentheses left again, then shifting the outer parenthesis right, with each step allowed by associativity. By a similar method (of "walking" the parenthesis all over the reduction path one step at a time), any two reduction paths (of the same list) can be shown equivalent.

# Implications

An associative function $f : X \times X \to X$ gives rise to a single natural function $f_n$ for reducing a list of $n$ inputs to a single output, given any number $n \ge 1$. ($f_1$ is the function that uses zero applications of $f,$ and thus leaves the input untouched.) This means that we can see associative functions not as just binary functions, but as a family of functions for reducing lists to a single output.

Furthermore, whenever we want to use an associative function to reduce a list to a single element, we can do so in a "local" fashion, by repeatedly combining adjacent elements in the list, without worrying about the order in which we do so. This is certainly not true for all functions that reduce a list to a single output! For example, consider the function `adjacent_ones` which takes a list of ones and zeroes and produces a `1` if the list contains any adjacent ones, and a `0` otherwise. `adjacent_ones([https://arbital.com/p/0,0,0,1,1,0](https://arbital.com/p/0,0,0,1,1,0))=1`, but we can't compute this result by just combining adjacent elements pairwise willy-nilly: If we use the order ((00)(01))(10) we get the answer `0`, when the correct answer is `1`.

Associative functions are precisely those that give rise to a family of functions for reducing lists where the result _can_ be computed by just combining adjacent elements pairwise willy-nilly. Thus, associative operators describe list operations that can be computed _locally,_ only ever looking at two adjacent elements, without worrying that some important global structure (such as "are there two ones in a row?") will be destroyed.

This family of functions also has the nice property that if you reduce one list $[b, c, \ldots](https://arbital.com/p/a,)$ to a single element $\alpha,$ and another list $[y, z, \ldots](https://arbital.com/p/x,)$ to a single element $\chi,$ then $f(\alpha, \chi)$ is the same as reducing $[b, c, \ldots, x, y, z, \ldots](https://arbital.com/p/a,):$ in other words, we can reduce two lists separately and then combine the results, and get the same answer as we would have gotten by combining the lists first and then reducing them to a single result.

# What about empty lists?

Any associative function $f$ can be used to reduce a non-empty finite list to a single element, by repeatedly applying $f$ to adjacent elements, without worrying about the order. This works for one-element lists, despite the fact that $f$ takes two parameters, because for one-element lists we can just use zero applications of $f$. But what about zero-element lists?

To create a family $f_n : X^n \to X$ of functions on lists with the nice locality properties discussed above for all $n \ge 0,$ we need some "canonical" element of $0_X$ in $X$ that $f_0$ can return given an empty list, such that $0_X$ "acts like an empty element" with respect to $f.$ [Monoids](https://arbital.com/p/3h3) formalize this notion of "an associative operator plus an element that acts like an empty element," and thus, they capture the idea of a family of functions that can reduce any list (of finite size) to a single output, regardless of what order adjacent elements are combined in.

Given an [associative](https://arbital.com/p/3h4) [operator](https://arbital.com/p/3h7) $\cdot$ and a [list](https://arbital.com/p/3kd) $[b, c, \ldots](https://arbital.com/p/a,)$ of parameters, all ways of reducing the list to a single element by combining adjacent pairs yield the same result. In other words, while associativity only talks about the behavior of $f$ when combining three elements, this is sufficient to ensure that $f$ has similar good behavior when combining arbitrarily many elements. Thus, we can unambiguously drop parenthesis when using $\cdot$ to combine multiple elements. In other words, an associative function $f : X \times X \to X$ gives rise to a method for combining any non-empty finite list of elements of $X$ into a single output, such that the result can be computed by combining adjacent elements in any order.


Associativity: Examples

[https://arbital.com/p/toc:](https://arbital.com/p/toc:)

## Positive examples

### Addition

$(x + y) + z = x + (y + z)$ for all numbers $x, y,$ and $z.$ Thus, addition associates. One easy way to see this fact is to consider a [physical system](https://arbital.com/p/3mb) that implements addition, e.g., by taking two piles of poker chips (where a poker chip with $n$ chips represents the number $n$) in on two input belts, and producing a pile of poker chips on the output belt. This function can be implemented by simply shoving both input piles onto the output pile. Clearly, when combining three piles, it doesn't matter which piles get shoved onto the output tape in which order, so addition is associative.

### Multiplication

$(x \times y) \times z = x \times (y \times z)$ for all numbers $x, y,$ and $z.$ Thus, multiplication associates. It is, however, a little harder to see why this must be the case for multiplication.

Imagine a physical system implementing multiplication, by taking stacks of poker chips as input (where stacks with $n$ chips in them represents the number $n$), and producing an output by putting a copy of the left stack on the output tape for every chip in the right stack. Imagine using this function to combine three stacks of poker chips, a blue stack (representing $x$), a yellow stack (representing $y$), and a red stack (representing $z$). We can either calculate $(x \times y) \times z$ or $x \times (y \times z).$ In both cases, the result will be a bunch of copies of the original blue $x$ stack. The question is, how many copies will there be in each case?

In the first case, the machine first puts $y$ copies of the blue stack in a line, and then makes $z$ copies of that line. In the second case, the machine first makes $z$ copies of the yellow stack. Each of those yellow stacks corresponds to one of the lines from the first case, and each yellow chip on each yellow stack corresponds to one of the blue stacks in the first case. A blue stack is placed on the output in the second case for each of those yellow chips, so the number of blue stacks in each case is the same.

### Concatenation

The concatenation of [strings](https://arbital.com/p/3jr) is another associative function: `concat("a",concat("b","c")) = concat(concat("a","b"),"c") = "abc"`.

Concatenation is an example of an associative function that is not [commutative](https://arbital.com/p/3jb): When reducing a list of strings to a single string, it doesn't matter what order you combine adjacent elements in, but it _does_ matter that you leave the elements in their original order.

In fact, any function that just sticks its inputs together (possibly with some extra stuff in the middle) is associative: A function which takes wooden blocks as input, and glues them together (with a small white block in the middle) is associative, because if you put in blue, yellow, and red blocks then you get a blue-white-yellow-white-red block out the end, regardless of the order you combine adjacent blocks in.

## Negative examples

### Functions with a history

The function `xconcat` that concatenates its inputs and puts an "x" on the front: `xconcat("A", xconcat("B","C")) = "xAxBC"`, but `xconcat(xconcat("A", "B"), "C") = "xxABC"`. The problem in this case is that the output carries a trace of which adjacent elements were combined in which order, which makes the function non-associative.

In fact, any function that carries a history of the application order with it can't be associative. Thus, if the inputs carry history about which inputs were combined with which other inputs, and the output preserves (and adds to) that history, the function can't be associative. Associativity is, fundamentally, about the output not depending on which path was taken to get to it.

### Subtraction

Subtraction does not associate, as $(5-3)-2=0$ but $5-(3-2)=4.$ To see what went wrong, first notice that all inputs contribute either positively or negatively to the result. Label all inputs (in their original positive states) with up-arrows; we will track whether that input has a positive or negative impact on the final output. Subtraction flips its right-hand input upside down before combining it with its left-hand input, so given inputs labeled $\uparrow$ and $\uparrow$ we should label its output $\uparrow\downarrow.$ When combining three inputs, we can either combine the left two first or the right two first. If we combine the left two first, then the second subtraction is run on inputs labeled $\uparrow\downarrow$ and $\uparrow,$ and produces an output $\uparrow\downarrow\downarrow,$ in which the first input contributes positively and the other inputs contribute negatively. But if we combine the right two inputs first, then the second subtraction is run on inputs labeled $\uparrow$ and $\uparrow\downarrow,$ and produces an output $\uparrow\downarrow\uparrow,$ in which the first and third inputs contribute positively and the second contributes negatively. Thus, the contribution of the third input depends on which adjacent elements are combined in which order, so subtraction does not associate.

### Cyclical preferences

Consider a person who is playing a game that works as follows. The board has three positions: red, green, and blue. The player's objective is to complete as many clockwise red-green-blue cycles as possible, without ever backtracking in the counter-clockwise direction.

![3-cycle board](http://i.imgur.com/bxCnXUs.png)

 Each turn, the game offers them a choice of one of the three spaces, and they get to choose whether or not to travel to that square or stay where they are. Clearly, their preferences depend on where they currently are: If they're on "red", "green" is a good move and "blue" is a bad one; but if they're on "blue" then choosing "green" is ill-advised.

We can consider a binary operation $?$ which takes their current position on the left and the proposed position on the right, and returns the position that the player prefers. Specifically:

\begin{align}
& red \ ?\  red \ &= red\\
& red \ ?\  green \ &= green\\
& red \ ?\  blue \ &= red\\
\end{align}
\begin{align}
& green \ ?\  red \ &= green\\
& green \ ?\  green \ &= green\\
& green \ ?\  blue \ &= blue\\
\end{align}
\begin{align}
& blue \ ?\  red \ &= red\\
& blue \ ?\  green \ &= blue\\
& blue \ ?\  blue \ &= blue
\end{align}

This function does not associate, because $(red\ ?\ green)\ ?\ blue = blue$ but $red\ ?\ (green\ ?\ blue)=red.$ To show that a function is not associative, it is sufficient to simply do out the whole function table and then find any one case that violates the axiom of associativity, as above.

Yes: [https://arbital.com/p/Addition](https://arbital.com/p/Addition), [https://arbital.com/p/multiplication](https://arbital.com/p/multiplication), [string concatenation](https://arbital.com/p/3jv). No: [https://arbital.com/p/subtraction](https://arbital.com/p/subtraction), [https://arbital.com/p/division](https://arbital.com/p/division), a [https://arbital.com/p/-3jy](https://arbital.com/p/-3jy) that concatenates strings and then puts an "x" on the front.


Commutativity: Examples

## Positive examples

### Addition

$x+y = y+x$ for all numbers $x$ and $y,$ so addition commutes. One easy way to see this fact is to consider a physical system that implements addition, e.g., by taking two piles of poker chips (where a poker chip with nn chips represents the number nn) in on two input belts, and producing a pile of poker chips on the output belt. This function can be implemented by simply shoving both input piles onto the output pile. Because a pile of chips represents the same number regardless of how those chips are arranged on the output belt, it doesn't matter which pile comes in on which input belt, so addition is commutative.

### Multiplication

$x \times y = y \times x$ for all numbers $x$ and $y,$ so multiplication also commutes. It is a bit harder to see why this must be so in the case of multiplication, as $x \times y$ can be interpreted as "take the $x$ input and stretch it out according to the $y$ input," and it's not entirely obvious that "stretching the $x$ input according to $y$" should yield the same result as "stretching the $y$ input according to $x.$"

To see that multiplication commutes, it is helpful to visualize _copying_ $x$ (rather than stretching it out) into $y$ many rows. This allows us to visualize $x \times y$ as a square divided into $x$ parts along one side and $y$ parts along the other side, which makes it clearer that multiplication commutes.

![Multiplicaiton commutes](http://i.imgur.com/Q3QBUT6.png)

It is worth noting that the concept of "multiplication" generalizes beyond the concept of "numbers," and in those contexts, "stretching $x$ out according to $y$" and "stretching $y$ out according to $x$" might be quite different operations. For example, [multiplication of matrices](https://arbital.com/p/matrix_multiplication) does not commute.

### Maximum and minimum

The [max](https://arbital.com/p/maximum_function) and [min](https://arbital.com/p/minimum_function) functions commute, because "which of these elements is largest/smallest?" is a question that doesn't depend upon the order that the elements are presented in.

### Rock, Paper, Scissors

The question of who won in a game of [rock-paper-scissors](https://arbital.com/p/rock_paper_scissors) is commutative. Writing $r$ for rock, $p$ for paper, $s$ for scissors, and $?$ for the binary operator saying who won, we have $r ? p = p,$ $r ? s = r,$ and $p ? s = s,$ and so on. This function is commutative: who won a game of rock-paper-scissors does not depend upon which side of the judge they were standing on.

This provides an example of a commutative function that is not [associative](https://arbital.com/p/3h4): $r?p=p?r$ and so on, but $(r?p)?s=s$ while $r?(p?s)=r.$ In other words, who won a game of rock-paper-scissors does not depend on which side of the judge they were standing on, but if you have a line of people throw "rock," "paper," or "scissors," then the question "who won?" _does_ depend on which end of the line you start from.

## Negative examples

### Concatenation

Any function that includes its inputs in its outputs _in order_ is not commutative. For example, the function `pair` which puts its inputs into a pair (such that `pair(x, y) = (x, y)`) is not commutative. The function [concat](https://arbital.com/p/3jv), which also sticks its inputs together in order, is also not commutative, though it is associative.

### Division

Division is not commutative, because $x / y$ does not equal $y / x$ in general. In this case, the function is using $x$ and $y$ in a very different way. One is interpreted as a numerator that will be cut up, the other is treated as a denominator that says how many times to cut the numerator up. Numerator and denominator are not exchangeable concepts: Division needs to know which input is intended to be the numerator and which is intended to be the denominator.

### Matrix multiplication

[Matrix multiplication](https://arbital.com/p/matrix_multiplication) is not only non-commutative, it may fail to even make sense. Two [matrices](https://arbital.com/p/linear_algebra_matrix) can be multiplied if and only if the number of columns in the left-hand matrix is equal to the number of rows in the right-hand matrix. Thus, a $2 \times 3$ matrix can be multiplied with a $3 \times 5$ matrix if and only if the $2 \times 3$ matrix is on the left. Clearly, then, matrix multiplication does not commute.

Yes: addition, multiplication, maximum, minimum, rock-paper-scissors. No: subtraction, division, string concatenation, matrix multiplication.


Associativity vs commutativity

[Associativity](https://arbital.com/p/3h4) and [commutativity](https://arbital.com/p/3jb) are often confused, because they are both constraints on how a [function](https://arbital.com/p/3jy) treats its inputs, and they are both constraints saying "these two paths for calculating the output must yield the same answer." The difference is that commutativity says "this function shouldn't care what order its own arguments come in," while associativity says "different ways of using this function multiple times should yield the same results."

Commutativity is about an invariance in _one_ call to the function: If you run the function once with inputs $x$ and $y,$ you'll get the same result as if you run it once with inputs $y$ and $x.$

Associativity is about an invariance in _multiple_ calls to the function: If you combine four elements using three function calls in the order $a \cdot (b \cdot (c \cdot d)),$ you'll get the same answer as if you use the three function calls in the order $((a \cdot b) \cdot c) \cdot d.$

Commutativity means you can swap the order of the elements. Associativity means you can drop parenthesis when combining multiple elements in a row.

If an operator $\cdot$ is both commutative and associative, then when you're combining multiple elements in a row, you're welcome to combine them in _any_ order: For example, when adding $3 + 2 + (-7) + 5 + (-2) + (-3) + 7,$ we can re-arrange the list to get $3 - 3 + 2 - 2 + 7 - 7 + 5 = 5,$ which makes calculation easy. To see that this is the case, note that the result won't change if we swap any two adjacent elements, because associativity means that we can pick any two adjacent elements to start with and commutativity says that the order of those elements doesn't matter. Then, we can re-arrange the list however we like using a series of two-element swaps.
[Perhaps conditionalize this parenthetical on something like "cares about abstract algebra" or "wants tidbits they don't understand to point them towards cool things" or something.](https://arbital.com/p/fixme:)
(This is why [commutative groups](https://arbital.com/p/3h2) are especially easy to work with.)

## Examples

Addition is both commutative and associative (same with multiplication). Subtraction is neither commutative nor associative (same with division).

[String concatenation](https://arbital.com/p/3jv) is associative but not commutative: If you're sticking a bunch of [strings](https://arbital.com/p/3jr) together end on end, then it doesn't matter which adjacent pairs you combine first, but `"onetwo"` is a very different string than `"twoone"`.

Rock-paper-scissors is commutative but not associative. The winner of "rock v scissors" is rock, and the winner of "scissors v rock" is rock, and so on; but if you have a bunch of people in a line make rock, paper, or scissors signs, then who is the winer depends on which end of the line you start from: If the line has three people, and they throw [paper, scissors](https://arbital.com/p/rock,), then the winner will be whoever threw scissors if you start from the left (by finding the winner between rock and paper,  and then playing that against scissors) or rock if you start on the right (by finding the winner between paper and scissors, and then playing that against rock).

The function `pair` which puts its inputs into a pair (such that `pair(x, y) = (x, y)`) is neither commutative nor associative: `(x, y)` does not equal `(y, x)`, and `(x, (y, z))` does not equal `((x, y), z)`. The output of `pair` preserves the ordering and structure of all its inputs, and leaves a trace that allows one to distinguish which inputs it was called on when. Both commutativity and associativity require the function to be "forgetful" about some aspect of how it was called: Commutative functions need to be insensitive to the ordering of their arguments; associative functions need to leave no trace that can be used to figure out which inputs were combined in what order. Thus, any function that bakes a lot of history about what it was called with into the output is unlikely to be commutative or associative.

## Mnemonics

If it helps you to remember which is which, consider these two mnemonics:

For "commutativity," imagine that the two parameters to the function each live on one side of town and work on the other side. Each morning, they pass each other on their morning _commute,_ as they swap places.

For "associativity," imagine a bunch of _associates_ all standing in a line, ranked according to their hierarchy in a law firm. Any two adjacent people are willing to converse with each other, although people on one end of the line might not be willing to associate with people on the other end.


Eigenvalues and eigenvectors

Consider a linear transformation represented by a matrix $A$, and some vector $v$. If $Av = \lambda v$, we say that $v$ is an _eigenvector_ of $A$ with corresponding eigenvalue $\lambda$. Intuitively, this means that $A$ doesn't rotate or change the direction of $v$; it can only stretch it ($|\lambda| > 1$) or squash it ($|\lambda| < 1$) and maybe flip it ($\lambda < 0$). While this notion may initially seem obscure, it turns out to have many useful applications, and many fundamental properties of a linear transformation can be characterized by its eigenvalues and eigenvectors.

Alexei Andreev
Eric Rogstad
Zack M. Davis
Nate Soares
Ryan Neufeld
Qiaochu Yuan
Eric Bruylant


Logarithm

summary(Technical): $\log_b(n)$ is defined to be the number $x$ such that $b^x = n.$ Thus, logarithm functions satisfy the following properties, among others:

- $\log_b(1) = 0$
- $\log_b(b) = 1$
- $\log_b(x\cdot y) = log_b(x) + \log_b(y)$
- $\log_b(\frac{x}{y}) = \log_b(x) - \log_b(y)$
- $\log_b(x^n) = n\log_b(x)$
- $\log_b(\sqrt[https://arbital.com/p/n](https://arbital.com/p/n){x}) = \frac{\log_b(x)}{n}$
- $\log_b(n) = \frac{\log_a(n)}{\log_a(b)}$

summary(Inverse exponentials): Logarithms are the inverse of [exponentials](https://arbital.com/p/exponential). That is, for any base $b$ and number $n,$ $\log_b(b^n) = n$ and $b^{\log_b(n)} = n.$

summary(Measure of data): A message that singles out one thing from a set of $n$ carries $\log(n)$ units of data, where the unit of information depends on the base of the logarithm. For example, a message singling out one thing from 1024 carries about three [decits](https://arbital.com/p/3wn) of data (because $\log_{10}(1024) \approx 3$), or exactly ten [bits](https://arbital.com/p/3p0) of data (because $\log_2(1024)=10$). For details, see [https://arbital.com/p/+3p0](https://arbital.com/p/+3p0).

summary(Generalized lengths):  A quick way of approximating the logarithm base 10 is to look at the length of a number: 103 is a 3-digit number but it's _almost_ a 2-digit number, so its logarithm (base ten) is a little higher than 2 (it's about 2.01). 981 is also a three-digit number, and it's using nearly all three of those digits, so its logarithm (base ten) is just barely lower than 3 (it's about 2.99). In this way, logarithms generalize the notion of "length," and in particular, $\log_b(n)$ measures the generalized length of the number $n$ when it's written in $b$-ary notation.

The logarithm base $b$ of a number $n,$ written $\log_b(n),$ is the answer to the question "how many times do you have to multiply 1 by $b$ to get $n$?" For example, $\log_{10}(100)=2,$ and $\log_{10}(316) \approx 2.5,$ because $316 \approx$ $10 \cdot 10 \cdot \sqrt{10},$ and [multiplying by $\sqrt{10}$ corresponds to multiplying by 10 "half a time"](https://arbital.com/p/).

In other words, $\log_b(x)$ counts the number of $b$-factors in $x$. For example, $\log_2(100)$ counts the number of "doublings" in the number 100, and $6 < \log_2(100) < 7$ because scaling an object up by a factor of 100 requires more than 6 (but less than 7) doublings. For an introduction to logarithms, see the [Arbital logarithm tutorial](https://arbital.com/p/3wj). For an advanced introduction, see the [advanced logarithm tutorial](https://arbital.com/p/advanced_log_tutorial).

Formally, $\log_b(n)$ is defined to be the number $x$ such that $b^x = n,$ where $b$ and $n$ are numbers. $b$ is called the "base" of the logarithm, and has a relationship to the [base of a number system](https://arbital.com/p/number_base). For a discussion of common and useful bases for logarithms, see the page on [logarithm bases](https://arbital.com/p/logarithm_bases). $x$ is unique if by "number" we mean [$\mathbb R$](https://arbital.com/p/4bc), but may not be unique if by "number" we mean [$\mathbb C$](https://arbital.com/p/complex_number). For details, see the page on [complex logarithms](https://arbital.com/p/complex_logarithm).

# Basic properties

Logarithms satisfy a number of desirable properties, including:

- $\log_b(1) = 0$ for any $b$
- $\log_b(b) = 1$ for any $b$
- $\log_b(x\cdot y) = log_b(x) + \log_b(y)$
- $\log_b(x^n) = n\log_b(x)$
- $\log_a(n) = \frac{\log_b(n)}{\log_b(a)}$

For an expanded list of properties, explanations of what they mean, and the reasons for why they hold, see [https://arbital.com/p/+3wp](https://arbital.com/p/+3wp).

# Interpretations

- Logarithms can be interpreted as a generalization of the notion of the [length of a number](https://arbital.com/p/number_length): 103 and 981 are both three digits long, but, intuitively, 103 is only barely using three digits, whereas 981 is pushing its three digits to the limit. Logarithms quantify this intuition: the [common logarithm](https://arbital.com/p/common_logarithm) of 103 is approximately 2.01, and the common log of 981 is approximately 2.99. Logarithms give rise to a notion of exactly how many digits a number is "actually" making use of, and give us a notion of "fractional digits." For more on this interpretation (and why it is 316, not 500, that is two and a half digits long), see [https://arbital.com/p/+416](https://arbital.com/p/+416).

- Logarithms can be interpreted as a measure of how much data it takes to carry a message. Imagine that you and I are both facing a collection of 100 different objects, and I'm thinking of one of them in particular. If I want to tell you which one I'm thinking of, how many digits do I need to transmit to you? The answer is $\log_{10}(100)=2,$ assuming that by "digit" we mean "some method of encoding one of the symbols 0-9 in a physical medium." Measuring data in this way is the cornerstone of [information theory](https://arbital.com/p/3qq).

- Logarithms are the inverse of exponentials. The function $\log_b(\cdot)$ inverts the function $b^{\ \cdot}.$ In other words, $\log_b(n) = x$ implies that $b^x = n,$ so $\log_b(b^x)=x$ and $b^{\log_b(n)}=n.$ Thus, logarithms give us tools for analyzing anything that grows exponentially. If a population of bacteria doubles each day, then logarithms measure days in terms of bacteria — that is, they can tell you how long it will take for the population to reach a certain size. For more on this idea, see [https://arbital.com/p/+3wr](https://arbital.com/p/+3wr).

# Applications

Logarithms are ubiquitous in many fields, including mathematics, physics, computer science, cognitive science, and artificial intelligence, to name a few. For example:

- In mathematics, the most natural logarithmic base is [$e$](https://arbital.com/p/mathematics_e) ([Why?](https://arbital.com/p/log_e_is_natural)) and the log base $e$ of $x$ is written $\ln(x)$, pronounced "[natural log](https://arbital.com/p/natural_logarithm) of x." The natural logarithm of a number gives one notion of the "intrinsic length" of a number, a concept that proves useful when reasoning about other properties of that number. For example, the quantity of prime numbers smaller than $x$ is approximately $\frac{x}{\ln(x)},$ this is the [prime number theorem](https://arbital.com/p/prime_number_theorem).

- Logarithms also give us tools for measuring the runtime (or memory usage) of algorithms. When an algorithm uses a [divide and conquer](https://arbital.com/p/divide_and_conquer) approach, the amount of time (or memory) used by the algorithm increases logarithmically as the input size grows linearly. For example, the amount of time that it takes to perform a [binary search](https://arbital.com/p/binary_search) through $n$ possibilities is $\log_2(n),$ which means that the search takes one unit longer to run every time the set of things to search through doubles in size.

- Logarithms give us tools for studying the tools we use to represent numbers. For example, humans tend to use ten different symbols to represent numbers (0, 1, 2, 3, 4, 5, 6, 7, 8, and 9), while computers tend to use two digits (0 and 1). Are some representations better or worse than others? What are the pros and cons of using more or fewer symbols? For more on these questions, see [Number bases](https://arbital.com/p/number_base).

- The human brain encodes various perceptions logarithmically. For example, the perceived tone of a sound goes up by one octave every time the frequency of air vibrations doubles. Your perception of tone is proportional to the logarithm (base 2) of the frequency at which the air is vibrating. See also [Hick's law](https://arbital.com/p/https://en.wikipedia.org/wiki/Hick%27s_law).

The logarithm base $b$ of a number $n,$ written $\log_b(n),$ is the answer to the question "how many times do you have to multiply 1 by $b$ to get $n$?" For example, $\log_{10}(1000)=3,$ because $10 \cdot 10 \cdot 10 = 1000,$ and $\log_2(16)=4$ because $2 \cdot 2 \cdot 2 \cdot 2 = 16.$


Relation

%%comment:
I do not want to be shortened. The motivation for this is that I would prefer that someone has the ability to learn everything they need to know about relations just by reading the popup summary.
%%

summary: A **relation** is a [set](https://arbital.com/p/3jz) of [tuples](https://arbital.com/p/tuple_mathematics), all of which have the same [arity](https://arbital.com/p/tuple_arity). The inclusion of a tuple in a relation indicates that the components of the tuple are related. A set of $n$-tuples is called an $n$*-ary relation*. Sets of pairs are called binary relations, sets of triples are called ternary relations, etc.

Examples of binary relations include the equality relation on natural numbers $\{ (0,0), (1,1), (2,2), ... \}$ and the predecessor relation $\{ (0,1), (1,2), (2,3), ... \}$. When a symbol is used to denote a specific binary relation ($R$ is commonly used for this purpose),  that symbol can be used with infix notation to denote set membership: $xRy$ means that the pair $(x,y)$ is an element of the set $R$.

A **relation** is a [set](https://arbital.com/p/3jz) of [tuples](https://arbital.com/p/tuple_mathematics), all of which have the same [arity](https://arbital.com/p/tuple_arity). The inclusion of a tuple in a relation indicates that the components of the tuple are related. A set of $n$-tuples is called an $n$*-ary relation*. Sets of pairs are called binary relations, sets of triples are called ternary relations, etc.

Examples of binary relations include the equality relation on natural numbers $\{ (0,0), (1,1), (2,2), ... \}$ and the predecessor relation $\{ (0,1), (1,2), (2,3), ... \}$. When a symbol is used to denote a specific binary relation ($R$ is commonly used for this purpose),  that symbol can be used with infix notation to denote set membership: $xRy$ means that the pair $(x,y)$ is an element of the set $R$.

Kevin Clancy
Alexei Andreev
Dylan Hendrickson
Nate Soares
Eric Bruylant
Joe Zeng


Bit (of data)

A bit of data (not to be confused with a [https://arbital.com/p/shannon](https://arbital.com/p/shannon), an [abstract bit](https://arbital.com/p/3y3), or a [bit of evidence](https://arbital.com/p/evidence_bit)) is the amount of data required to single out one message from a set of two. If you want to single out one of the hands of one of your biological grandparents (such as the left hand of your paternal grandmother), you can do that with three bits of data: One to single out "left hand" or "right hand", one to single out "maternal" or "paternal", and one to single out "grandfather" or "grandmother". In order for someone to look at the data and know which hand you singled out, they need to know what method you used to [encode](https://arbital.com/p/encoding) the message into the data.

Data can be stored in physical systems which can be put into multiple states: A coin can be used to store a single bit of data (by placing it either heads or tails); a normal six-sided die can be used to store a little over [2.5 bits](https://arbital.com/p/3ty) of data. In general, a message that singles out one thing from a set of $n$ is defined to contain [$\log_2](https://arbital.com/p/3nd) bits of data.

"Bit" is a portmanteau of "binary digit": "binary" because it is the amount of information required to single out one message from a set of 2. The 2 is arbitrary; analogous units of data exist for every other number. For example, a [https://arbital.com/p/3ww](https://arbital.com/p/3ww) is the amount of data required to single out one message from a set of three, a [https://arbital.com/p/3wn](https://arbital.com/p/3wn) is the amount of data it takes to single out one message from a set of ten, and so on. It is easy to convert between units, for example, a decit is $\log_2(10) \approx 3.32$ bits, because it takes a little over three bits to pick one thing out from a set of 10. See also the pages on [converting between units of data](https://arbital.com/p/) and [fractional bits](https://arbital.com/p/3ty).

[Talk about how we want to define data such that two objects hold twice as much.](https://arbital.com/p/fixme:)

The amount of data you can transmit (or store) grows [exponentially](https://arbital.com/p/exponential) in the number of bits at your disposal. For example, consider a punch card that can hold ten bits of data. You can use one punch card to pick out a single thing from a set of $2^{10}=1024.$ From two punch cards you can pick out a single thing from a set of $2^{20}=1048576.$ The number of _things you can distinguish_ using two punch cards is 1024 times larger than the number of things you can distinguish with one punch card, and the amount of data you can encode using two punch cards is precisely twice as much (20 bits) as the amount of information you can encode using one punch card (10 bits). In other words, you can single out one object from a collection of $n$ (or store a number between 1 and $n$) using [$\log_2](https://arbital.com/p/3nd) bits of data. For more on why the amount of data is logarithmic in the number of possible messages, see [Data is logarithmic](https://arbital.com/p/).

[Add a formalization section.](https://arbital.com/p/fixme:)

A bit of data is the amount of data required to single out one message from a set of two. Equivalently, it is the amount of data required to cut the set of possible messages in half. If you want to single out one of the hands of one of your biological grandparents (such as the left hand of your paternal grandmother), you can do that by transmitting three bits of data: One to single out "left hand" or "right hand", one to single out "maternal" or "paternal", and one to single out "grandfather" or "grandmother". We can also speak of storing data in physical systems which can be put into multiple states: A coin can be used to store a single bit of data (by placing it either heads or tails); a normal six-sided die can be used to store a little over [2.5 bits](https://arbital.com/p/3ty) of data. In general, a message that singles out one message from a set of $n$ is defined to contain [$\log_2](https://arbital.com/p/3nd) bits of data.


Information theory

The study (and quantificaiton) of information, and its communication and storage.

Eric Bruylant
Nate Soares
Eliezer Yudkowsky
jorgejuan lafuente


Partially ordered set

A **partially ordered set** (also called a **poset**) is a pair $\langle P, \leq \rangle$ of a set $P$ and a [binary relation](https://arbital.com/p/3nt) $\leq$ on $P$ such that for all $p,q,r \in P$, the following properties are satisfied:

- [Reflexivity](https://arbital.com/p/5dy): $p \leq p$
- [Transitivity](https://arbital.com/p/573): $p \leq q$ and  $q \leq r$ implies $p \leq r$
- [Anti-symmetry](https://arbital.com/p/5lt): $p \leq q$ and $q \leq p$ implies $p = q$

$P$ is referred to as the poset's underlying set and $\leq$
 is referred to as its order relation. Posets are the central object of study in [order theory](https://arbital.com/p/3dt).

Notations and fundamentals
=======================

Greater than
--------------------

A greater than relation $\geq$ can be derived as the transpose of a poset's less than relation: $a \geq b$ means that $b \leq a$.

Incomparability
-------------------------

For poset elements $p$ and $q$, if neither $p \leq q$ nor $q \leq p$ then we say that $p$ and $q$ are *incomparable*, and write $p \parallel q$.

Strict orders
--------------------

From any poset $\langle P, \leq \rangle$, we can derive an underlying strict order $<$. For $p, q \in P$, $p < q$ means that the following conditions are satisfied:

- 1.) $p \leq q$
- 2.) $p \neq q$

The cover relation
-----------------------------

From any poset $\langle P, \leq \rangle$, we can derive an underlying *cover relation* $\prec$, defined such that for $p,q \in P$, $p \prec q$ whenever the following two conditions are satisfied:

- 1.) $p < q$
- 2.) For all $r \in P$, $p \leq r < q$ implies $p = r$

Simply put, $p \prec q$ means that $q$ is the smallest element of $P$ which is strictly greater than $p$.
$p \prec q$ is pronounced "$p$ is covered by $q$", or "$q$ covers $p$", and $q$ is said to be a *cover* of $p$.

Hasse diagrams
--------------------------

Let $\langle P, \leq \rangle$ be the poset such that $P = \{ p, q, r \}$ and $\leq = \{(p,p),(p,q)(p,r),(q,q),(q,r),(r,r) \}$. The order relation $\leq$, being binary, can be depicted as a directed graph, though the resulting image leaves something to be desired.

![Naive first attempt at representing P with a directed graph](http://i.imgur.com/fOkTwXW.png)

%%%comment:
dot source:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width)
  rankdir = BT;
  p -> q;
  q -> r;
  p -> p;
  r -> r;
  q -> q;
  p -> r;
}
%%%

Including an edge for every pair in $\leq$ results in a rather noisy looking graph. In particular, as long as we are under assumption that the graph we are viewing depicts a poset, placing a reflexive edge on each node is tedious and redundant. Our first step toward cleaning up this graph is to remove all reflexive edges, instead leaving reflexivity as an implicit assumption. Likewise, we can leave those edges which are implied by transitivity — $(p,r)$ is the only such edge in this poset — implicit. As a final step, we remove the arrow heads from our edges, leaving their directions implied by the y-coordinates of the nodes they connect: an edge between two nodes means that the poset element corresponding to the lower node is less than the poset element corresponding to the higher node. The simplified depiction of $\langle P, \leq \rangle$ then follows.

![Hasse diagram for P](http://i.imgur.com/W8erVrC.png)

%%%comment:
dot source:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width);
  edge [= "none"](https://arbital.com/p/arrowhead);
  rankdir = BT;
  p -> q;
  q -> r;
}
%%%

Such a depiction is called a *Hasse diagram*; drawing a Hasse diagram is the standard method for visually depicting a poset. Now that we understand what Hasse diagrams are, we can provide a concise definition: a Hasse diagram is a graph-based visual depiction of a poset $\langle P, \leq \rangle$, where elements of $P$ are represented as nodes, and for all $(p,q) \in P^2$ such that $p \prec q$, an upward edge is drawn from $p$ to $q$.

$p \leq q$ requires that $p$ must appear lower in a Hasse diagram than $q$. However, the converse is not true.  In the following Hasse diagram, $t \parallel r$ even though $t$ is positioned lower than $r$.

![t is lower than r but incomparable to r](http://i.imgur.com/hmyUZv9.png)

%%%comment:
dot source:

digraph G {
  node [= 0.1, height = 0.1](https://arbital.com/p/width);
  edge [= "none"](https://arbital.com/p/arrowhead);
  rankdir = BT;
  p -> t;
  p -> r;
  t -> q;
  q -> s;
  r -> s;
}
%%%

Duality
-----------

Every poset $\langle P, \leq \rangle$ has a corresponding *dual* poset $\langle P^{\partial}, \geq \rangle$, where $P^{\partial}$ (pronounced "P op") is a set whose elements are in correspondence with those of $P$, and $\geq$ is the transpose of $\leq$ discussed previously. The Hasse diagram of $P^{\partial}$ is obtained by flipping the Hasse diagram of $P$ upside down. Whenever $\phi$ is a propsition about posets, we obtain $\phi$'s *dual proposition* $\phi^{\partial}$ by replacing all occurences of $\leq$ with $\geq$ and all occurrences of $\geq$ with $\leq$.

The existence of dual posets and dual propositions gives rise to the *duality principle*: if a proposition $\forall P. \phi$, quantified over all posets, is true then its dual $\forall P. \phi^{\partial}$ is also true. The duality principle works because instantiating $\forall P. \phi$ with a poset $P$ is equivalent to instantiating $\forall P. \phi^{\partial}$ with $P^{\partial}$. This is due to the fact that $a \leq b$ in $P$ iff $a \geq b$ in $P^{\partial}$. Theorems involving posets often come with a free dual theorem thanks to the duality principle.

%%%knows-requisite([https://arbital.com/p/4c7](https://arbital.com/p/4c7)):
Poset as category
==============

Any poset $\langle P, \leq \rangle$ can be viewed as a [category](https://arbital.com/p/category) in which the objects are the elements of $P$, and for all $p,q \in P$, there is a unique morphism from $p$ to $q$ iff $p \leq q$. This category has closure under morphism composition due to the transitivity of $\leq$ and identity morphisms due to the reflexivity of $\leq$. Associativity of morphism composition stems from the uniqueness of arrows between any pair of objects. The functors between poset categories are [monotone maps](https://arbital.com/p/poset_monotone_map).
%%%

Additional Material
----------------------------------

For some additional examples of posets, see [https://arbital.com/p/43s](https://arbital.com/p/43s).
To test your knowledge of posets, try these [exercises](https://arbital.com/p/4l1).
The most useful operators on elements of a poset are called the [join and meet](https://arbital.com/p/3rc).
We can relate the elements of one poset to another through the use of [monotone functions](https://arbital.com/p/5jg).

Kevin Clancy
Eric Rogstad
Patrick Stevens
Nate Soares
Eric Bruylant
Joe Zeng

A partially ordered set (also called a poset) is a pair $\langle P, \leq \rangle$ of a set $P$ and a [binary relation](https://arbital.com/p/3nt) $\leq$ on $P$ such that for all $p,q,r \in P$, the following properties are satisfied:

- [Reflexivity](https://arbital.com/p/5dy): $p \leq p$
- [Transitivity](https://arbital.com/p/573): $p \leq q$ and  $q \leq r$ implies $p \leq r$
- [Anti-symmetry](https://arbital.com/p/5lt): $p \leq q$ and $q \leq p$ implies $p = q$

$P$ is referred to as the poset's underlying set and $\leq$
 is referred to as its order relation. When the order relation of a poset $\langle P, \leq \rangle$ is clear from context, the poset can be referred to merely as $P$. Posets are the central object of study in [order theory](https://arbital.com/p/3dt).


Join and meet

Let $\langle P, \leq \rangle$ be a [https://arbital.com/p/-3rb](https://arbital.com/p/-3rb), and let $S \subseteq P$. The **join** of $S$ in $P$, denoted by $\bigvee_P S$, is an element $p \in P$ satisfying the following two properties:

* p is an *upper bound* of $S$; that is, for all $s \in S$, $s \leq p$.
* For all upper bounds $q$ of $S$ in $P$, $p \leq q$.

$\bigvee_P S$ does not necessarily exist, but if it does then it is unique. The notation $\bigvee S$ is typically used instead of $\bigvee_P S$ when $P$ is clear from context. Joins are often called *least upper bounds* or *supremums*. For $a, b$ in $P$, the join of $\{a,b\}$ in $P$ is denoted by $a \vee_P b$, or $a \vee b$ when $P$ is clear from context.

The dual concept of the join is that of the meet. The **meet** of $S$ in $P$, denoted by $\bigwedge_P S$, is defined an element $p \in P$ satisfying.

* p is a *lower bound* of $S$; that is, for all $s$ in $S$, $p \leq s$.
* For all lower bounds $q$ of $S$ in $P$, $q \leq p$.

Meets are also called *infimums*, or *greatest lower bounds*. The notations $\bigwedge S$, $p \wedge_P q$, and $p \wedge q$ are all have meanings that are completely analogous to the aforementioned notations for joins.

Basic example
--------------------------

![Joins Failing to exist in a finite lattice](http://i.imgur.com/sx1Ss9w.png)

The above Hasse diagram represents a poset with elements $a$, $b$, $c$, and $d$. $\bigvee \{a,b\}$ does not exist because the set $\{a,b\}$ has no upper bounds. $\bigvee \{c,d\}$ does not exist for a different reason: although $\{c, d\}$ has upper bounds $a$ and $b$, these upper bounds are incomparable, and so $\{c, d\}$ has no *least* upper bound. There do exist subsets of this poset which possess joins; for example, $a \vee c = a$, $\bigvee \{b,c,d\} = b$, and $\bigvee \{c\} = c$.

Now for some examples of meets. $\bigwedge \{a, b, c, d\}$ does not exist because $c$ and $d$ have no common lower bounds. However, $\bigwedge \{a,b,d\} = d$ and $a \wedge c = c$.

Additional Material
---------------------------------

* [Examples](https://arbital.com/p/3v4)
* [Exercises](https://arbital.com/p/4ll)

Further reading
---------------
* [Lattices](https://arbital.com/p/46c)

Let $\langle P, \leq \rangle$ be a [poset](https://arbital.com/p/3rb), and let $S \subseteq P$. The **join** of $S$ in $P$, denoted by $\bigvee_P S$, is an element $p \in P$ satisfying the following two properties:

* p is an *upper bound* of $S$; that is, for all $s \in S$, $s \leq p$.
* For all upper bounds $q$ of $S$ in $P$, $p \leq q$.

$\bigvee_P S$ does not necessarily exist, but if it does then it is unique. The notation $\bigvee S$ is typically used instead of $\bigvee_P S$ when $P$ is clear from context. Joins are often called *least upper bounds* or *supremums*. For $a, b$ in $P$, the join of $\{a,b\}$ in $P$ is denoted by $a \vee_P b$, or $a \vee b$ when $P$ is clear from context. **Meets** are greatest lower bounds, and are related to joins by duality.


Start

Start-Class pages are visibly incomplete, or need significant work to be useful.  This covers pages missing essential information, pages without coherent structure, pages lacking [summaries](https://arbital.com/p/1kl) (or first paragraphs that serve as a summary), pages with sufficiently severe prose or stylistic issues, and any other problem that prevents pages from being useful to readers or fitting into their domains.  Start-Class pages have significant content, but are still in the category "should be seen by editors" rather than "should be seen by readers."

Contrast:

- [https://arbital.com/p/72](https://arbital.com/p/72), for tiny pages containing only a paragraph or a couple of sentences of text.
- [https://arbital.com/p/4gs](https://arbital.com/p/4gs), for pages which only contain a formal definition and don't try to explain the topic.

**Also** use the tag [https://arbital.com/p/4v](https://arbital.com/p/4v) if the page is being actively edited.

**[Quality scale](https://arbital.com/p/4yg)**

* [https://arbital.com/p/4ym](https://arbital.com/p/4ym)
* [https://arbital.com/p/4gs](https://arbital.com/p/4gs)
* [https://arbital.com/p/72](https://arbital.com/p/72)
* [https://arbital.com/p/3rk](https://arbital.com/p/3rk)
* [https://arbital.com/p/4y7](https://arbital.com/p/4y7)
* [https://arbital.com/p/4yd](https://arbital.com/p/4yd)
* [https://arbital.com/p/4yf](https://arbital.com/p/4yf)
* [https://arbital.com/p/4yl](https://arbital.com/p/4yl)

Kevin Clancy
Joe Zeng
Alexei Andreev
Eric Rogstad
Stephanie Zolayvar
Nate Soares
Eric Bruylant
Jaime Sevilla Molina
Eliezer Yudkowsky


Type theory

Hub page for type theory explanations


Group: Examples

# The symmetric groups

For every positive integer $n$ there is a group $S_n$, the [symmetric group](https://arbital.com/p/497) of order $n$, defined as the group of all permutations (bijections) $\{ 1, 2, \dots n \} \to \{ 1, 2, \dots n \}$ (or any other [https://arbital.com/p/-3jz](https://arbital.com/p/-3jz) with $n$ elements). The symmetric groups play a central role in group theory: for example, a [group action](https://arbital.com/p/3t9) of a group $G$ on a set $X$ with $n$ elements is the same as a [homomorphism](https://arbital.com/p/47t) $G \to S_n$.

Up to [conjugacy](https://arbital.com/p/4bj), a permutation is determined by its [cycle type](https://arbital.com/p/4cg).

# The dihedral groups

The [dihedral groups](https://arbital.com/p/4cy) $D_{2n}$ are the collections of symmetries of an $n$-sided regular polygon. It has a [presentation](https://arbital.com/p/5j9) $\langle r, f \mid r^n,  f^2,  (rf)^2 \rangle$, where $r$ represents rotation by $\tau/n$ degrees, and $f$ represents reflection.

For $n > 2$, the dihedral groups are non-commutative.

# The general linear groups

For every [field](https://arbital.com/p/481) $K$ and positive integer $n$ there is a group $GL_n(K)$, the [general linear group](https://arbital.com/p/general_linear_group) of order $n$ over $K$. Concretely, this is the group of all invertible $n \times n$ [matrices](https://arbital.com/p/matrix) with entries in $K$; more abstractly, this is the [automorphism group](https://arbital.com/p/automorphism) of a [vector space](https://arbital.com/p/3w0) of [dimension](https://arbital.com/p/vector_space_dimension) $n$ over $K$.

If $K$ is [algebraically closed](https://arbital.com/p/algebraically_closed_field), then up to conjugacy, a matrix is determined by its [Jordan normal form](https://arbital.com/p/Jordan_normal_form).

Daniel Satanove
Patrick Stevens
Qiaochu Yuan
Eric Bruylant
Mark Chimes

Examples of [groups](https://arbital.com/p/-3gd), including the [symmetric groups](https://arbital.com/p/-497) and [general linear groups](https://arbital.com/p/-general_linear_group).


Sample space

Motivation
===

If we are uncertain about some part of the world, then there are multiple different things that might happen there, as far as we know.
The sample space for a part of the world is the set of things that could possibly happen  there. We can use a sample space to reason using [probability theory](https://arbital.com/p/1bv), so that we assign probabilities to [events](https://arbital.com/p/event_probability) in a consistent way.

Definition
===

A sample space is a [set](https://arbital.com/p/3jz), usually denoted $\Omega$.

A sample space is the [set](https://arbital.com/p/3jz) $\Omega$ of possible ways that a part of the world might be, as far as you know.


Group theory: Examples

# Even and odd functions

Recall that a function $f : \mathbb{R} \to \mathbb{R}$ is [even](https://arbital.com/p/even_function) if $f(-x) = f(x)$, and [odd](https://arbital.com/p/odd_function) if $f(-x) = - f(x)$. A typical example of an even function is $f(x) = x^2$ or $f(x) = \cos x$, while a typical example of an odd function is $f(x) = x^3$ or $f(x) =   \sin x$.

We can think about evenness and oddness in terms of [group theory](https://arbital.com/p/3g8) as follows. There is a group called the [cyclic group](https://arbital.com/p/cyclic_group) $C_2$ of [order](https://arbital.com/p/3gg) $2$ acting on the set of functions $\mathbb{R} \to \mathbb{R}$: in other words, each element of the group describes a function of [type](https://arbital.com/p/3sz)

$$ (\mathbb{R} \to \mathbb{R}) \to (\mathbb{R} \to \mathbb{R}) $$

meaning that it takes as input a function $\mathbb{R} \to \mathbb{R}$ and returns as output another function $\mathbb{R} \to \mathbb{R}$.

$C_2$ has two elements which we'll call $1$ and $-1$. $1$ is the identity element: it acts on functions by sending a function $f(x)$ to the same function $f(x)$ again. $-1$ sends a function $f(x)$ to the function $f(-x)$, which visually corresponds to reflecting the graph of $f(x)$ through the y-axis. The group multiplication is what the names of the group elements suggests, and in particular $(-1) \times (-1) = 1$, which corresponds to the fact that $f(-(-x)) = f(x)$.

Any time a group $G$ [acts](https://arbital.com/p/3t9) on a set $X$, it's interesting to ask what elements are [invariant](https://arbital.com/p/invariant_under_a_group_action) under that group action. Here the invariants of functions under the action of $C_2$ above are the even functions, and they form a [https://arbital.com/p/subspace](https://arbital.com/p/subspace) of the [vector space](https://arbital.com/p/vector_space) of all functions.

It turns out that every function is uniquely the sum of an even and an odd function, as follows:

$$f(x) = \underbrace{\frac{f(x) + f(-x)}{2}}_{\text{even}} + \underbrace{\frac{f(x) - f(-x)}{2}}_{\text{odd}}.$$

This is a special case of various more general facts in [representation theory](https://arbital.com/p/3tn), and in particular can be thought of as the simplest case of the [discrete Fourier transform](https://arbital.com/p/discrete_Fourier_transform), which in turn is a [toy model](https://arbital.com/p/mathematical_toy_model) of the theory of [Fourier series](https://arbital.com/p/Fourier_series) and the [Fourier transform](https://arbital.com/p/Fourier_transform).

It's also interesting to observe that the cyclic group $C_2$ shows up in lots of other places in mathematics as well. For example, it is also the group describing how even and odd numbers add1 (where even corresponds to $1$ and odd corresponds to $-1$); this is the simplest case of [modular arithmetic](https://arbital.com/p/modular_arithmetic).

1That is: an even plus an even make an even, an odd plus an odd make an even, and an even plus an odd make an odd.


Programming in  Dependent Type Theory

All code in this article was written in the Lean theorem prover, which means you can copy any of it and paste it [here](https://leanprover.github.io/tutorial/?live) to try it out.

## Arithmetic and Interaction
While Lean is nominally an interactive theorem prover, much of the power of type theory comes from the fact that you can treat it like a programming language.

There are a few different commands for interacting with Lean.
The one I make the most use of is the `check` command, which prints out the type of the expression following it.
So, for example, `check 3` outputs `num`, and `check (3 : nat)` outputs `nat`.

We can also make definitions

    definition five : num := 3 + 2

This declares a new constant `five` of type `num` to which we give the value `3 + 2`.
We can define functions with similar syntax

    definition num.const₀ : num → num → num := λ x y, x -- this is a comment
    -- Lean can infer types whenever they're unique
    definition num.const₁                    := λ (x y : num), x
    -- we can also name the arguments in the definition rather than the function body
    definition num.const₂ (x y : num)        := x

The definition of polymorphic functions becomes the first point where we get a hint about what makes programming in dependent type theory different from, say, Haskell.
In dependent type theory, the term and type languages are unified, so in order to write a polymorphic function we must take the type as an argument.

    definition poly_id (A : Type) := λ (a : A), a
    -- or, equivalently
    definition poly_id₁ := λ A (a : A), a
    -- applied to arguments
    check poly_id num 1 -- num
    check poly_id (num → num → num) num.const -- num → num → num

Exercise: write a polymorphic version of `num.const₀`.

%%hidden(Show solution):
    definition poly_const (A B : Type) (a : A) (b : B) := a
%%

Having to explicitly indicate types everywhere is a pain.
In order to get around that, most proof assistants provide support for implicit arguments, which let you leave out arguments that only have one valid value.
In Lean, the syntax for implicits looks like this:

    definition id {A : Type} := λ (a : A), a

## Inductive types ##
Of course, none of this would be that useful if we couldn't define *new* types.
There are lots of ways to craft new types in dependent type theory, but among the most fundamental is the creation of inductive types.

To define a new inductive type, you give a list of constructor tags, each associated with a type representing the arguments it takes.
The simplest ones are just enumerations. For example, the days of the week:

    inductive weekday : Type :=
    | mon : weekday
    | tue : weekday
    | wed : weekday
    | thu : weekday
    | fri : weekday
    | sat : weekday
    | sun : weekday

This creates a new type `weekday`, and seven new constants (`weekday.mon`, `weekday.tue`, `weekday.wed`...) of type `weekday`.
If you're familiar with Haskell, you'll correctly notice that this looks an awful lot like GADT declarations.

Just like in Haskell, we can parametrize our types over other types, making new types like `either`:

    inductive either (A B : Type) : Type :=
    | inl {} : A → either A B``
    | inr {} : B → either A B

From this declaration, we get a new constant `either : Type → Type → Type`.
This represents the union of the types `A` and `B`, the type of values that belong either to `A` or `B`.

We can also define recursive types, such as natural numbers

    inductive nat : Type :=
    | zero : nat
    | succ : nat → nat

The easiest way to define functions over `nat`s is recursively.
For example, we can define addition as

    definition add (n : nat) : nat -> nat
    | nat.zero := n
    | (nat.succ m) := nat.succ (add m) -- n is constant at every recursive call

Bringing both of these together we can define the type of linked lists

    inductive list (A : Type) : Type :=
    | nil {} : list A
    | cons : A → list A → list A

We can also define functions over lists by pattern matching

    definition map {A B : Type} (f : A → B) : list A → list B
    | list.nil := list.nil
    | (list.cons x xs) := list.cons (f x) (map xs) -- f is constant at every recursive call

Exercise: write `foldr` and `foldl` by pattern matching

%%hidden(Show solution):
    definition foldr {A B : Type} (r : A → B → B) (vnil : B) : list A → B
    | list.nil := vnil
    | (list.cons x xs) := r x (foldr xs)

    definition foldl {A B : Type} (r : B → A → B) : B → list A → B :=
    | b list.nil := b
    | b (list.cons x xs) := foldl (r b x) xs
%%


Group action

An action of a [https://arbital.com/p/-3gd](https://arbital.com/p/-3gd) $G$ on a [https://arbital.com/p/-3jz](https://arbital.com/p/-3jz) $X$ is a function $\alpha : G \times X \to X$ ([colon-to notation](https://arbital.com/p/3vl)), which is often written $(g, x) \mapsto gx$ ([mapsto notation](https://arbital.com/p/3vm)), with $\alpha$ omitted from the notation, such that

1. $ex = x$ for all $x \in X$, where $e$ is the identity, and
2. $g(hx) = (gh)x$ for all $g, h \in G, x \in X$, where $gh$ implicitly refers to the group operation in $G$ (also omitted from the notation).

Equivalently, via [https://arbital.com/p/currying](https://arbital.com/p/currying), an action of $G$ on $X$ is a [group homomorphism](https://arbital.com/p/47t) $G \to \text{Aut}(X)$, where $\text{Aut}(X)$ is the [automorphism group](https://arbital.com/p/automorphism_group) of $X$ (so for sets, the group of all bijections $X \to X$, but phrasing the definition this way makes it natural to generalize to other [categories](https://arbital.com/p/category_theory)). It's a good exercise to verify this; Arbital [has a proof](https://arbital.com/p/49c).

Group actions are used to make precise the notion of "symmetry" in mathematics.

# Examples

Let $X = \mathbb{R}^2$ be the [Euclidean plane](https://arbital.com/p/Euclidean_geometry). There's a group acting on $\mathbb{R}^2$ called the [Euclidean group](https://arbital.com/p/Euclidean_group) $ISO(2)$ which consists of all functions $f : \mathbb{R}^2 \to \mathbb{R}^2$ preserving distances between two points (or equivalently all [isometries](https://arbital.com/p/isometry)). Its elements include translations, rotations about various points, and reflections about various lines.

Eric Rogstad
Patrick Stevens
Qiaochu Yuan


Probability distribution (countable sample space)

Definition
===

A probability distribution on a countable [https://arbital.com/p/3t4](https://arbital.com/p/3t4) $\Omega$ is a [https://arbital.com/p/3jy](https://arbital.com/p/3jy) $\mathbb{P}: \Omega \to [https://arbital.com/p/0,1](https://arbital.com/p/0,1)$ such that $\sum_{\omega \in \Omega} \mathbb{P}(\omega) = 1$.

Intuition
=====

We express a belief that "$x\in \Omega$ happens with probability $r$" by setting $\mathbb{P}(x) = r$. So a probability distribution divides up our anticipation of what will happen, out of the set $\Omega$ of things that might possibly happen.


Sample spaces are too large

When we are reasoning about an uncertain world using [probability theory](https://arbital.com/p/1bv), we have a [sample space](https://arbital.com/p/3t4) of possible ways the world could be, and we assign
probabilities to outcomes in the sample space with a [probability distribution](https://arbital.com/p/3tb). Unfortunately, often the sample space (which is supposed to contain every possible way the world might turn out) is very large and possibly infinite.

For example, if we are thinking about a box with some particles in it, our sample space is the enormous set of all possible arrangements of the particles.
To reason usefully about the box, we'd have to do computations using a gigantic table specifying a separate number for every single distinct arrangement of particles.
Two arrangements that are pretty much the same, as far as we care, take up just as much of our computational resources as two importantly different arrangements.
So, even though we could in principle reason consistently under uncertainty just using a probability distribution over a sample space, we cannot do so easily,
because we are trying to keep track of too much.

Sample spaces enumerate everything that could possibly happen, so that we can reason consistently under uncertainty. But this enumeration usually enumerates too many things, making it difficult to do computations using the raw sample space.


Group: Exercises

# Preliminaries

1. Show that the identity element in a group is unique. That is, if $G$ is a group and two elements $e_1, e_2 \in G$ both satisfy the axioms describing the identity element, then $e_1 = e_2$.

%%hidden(Show solution):
By definition, an identity element $e$ satisfies $eg = ge = g$ for all $g \in G$. Hence if $e_1$ is an identity, then $e_1 e_2 = e_2 e_1 = e_1$. And if $e_2$ is an identity, then $e_2 e_1 = e_1 e_2 = e_2$. Hence $e_1 = e_2$. Note that this argument makes no use of inverses, and so is valid for [monoids](https://arbital.com/p/3h3).
%%

2. Show that inverses are also unique. That is, if $g \in G$ is an element of a group and $h_1, h_2 \in G$ both satisfy the axioms describing the inverse of $g$, then $h_1 = h_2$.

%%hidden(Show solution):
By definition, an inverse $h$ of $g$ satisfies $hg = gh = e$. So $h_1 g = g h_1 = e$ and $h_2 g = g h_2 = e$. Hence, on the one hand,

$$h_1 g h_2 = (h_1 g) h_2 = (e) h_2 = h_2$$

and, on the other hand,

$$h_1 g h_2 = h_1 (g h_2) = h_1 (e) = h_1.$$

Hence $h_1 = h_2$.
%%

# Examples involving numbers

Determine whether the following sets equipped with the specified binary operations are groups. If so, describe their identity elements (which by the previous exercise must be unique) and how to take inverses.

1. The real numbers $\mathbb{R}$ together with the addition operation $(x, y) \mapsto x + y$.

%%hidden(Show answer):
Yes, this is a group. The identity element is $0$, and inverse is given by $x \mapsto -x$.
%%

2. The real numbers $\mathbb{R}$ together with the multiplication operation $(x, y) \mapsto xy$.

%%hidden(Show answer):
No, this is not a group. $0 \in \mathbb{R}$ has the property that $0 \times x = 0$ for all real numbers $x$, so it can't be invertible no matter what the identity is.
%%

3. The positive real numbers $\mathbb{R}_{>0}$ together with the multiplication operation $(x, y) \mapsto xy$.

%%hidden(Show answer):
Yes, this is a group. The identity is $1$, and inverse is given by $x \mapsto \frac{1}{x}$. In fact this group is [isomorphic](https://arbital.com/p/49x) to $(\mathbb{R}, +)$; can you name the isomorphism?
%%

4. The real numbers $\mathbb{R}$ together with the operation $(x, y) \mapsto x + y - 1$.

%%hidden(Show answer):
Yes, this is a group (in fact [isomorphic](https://arbital.com/p/49x) to $(\mathbb{R}, +)$; can you name the isomorphism?). The identity element is $1$, and inverse is given by $x \mapsto 2 - x$ (can you explain why, conceptually?).
%%

5. The real numbers $\mathbb{R}$ together with the operation $(x, y) \mapsto \frac{x + y}{1 + xy}$.

%%hidden(Show answer):
No, this is not a group. It's easy to be tricked into thinking it is, because if you just work through the algebra, it seems that all of the group axioms hold. However, this operation is not an operation! It's not defined if the denominator is $0$, because then we'd be [dividing by zero](https://arbital.com/p/division_by_zero).

This operation is interesting and useful, though, when it is defined. It shows up in [special relativity](https://arbital.com/p/special_relativity), where it describes how velocities add relativistically (in units where the speed of light is $1$).
%%

Patrick Stevens
Qiaochu Yuan
Eric Bruylant
Mark Chimes
Colin Benner

Test your understanding of the definition of a [group](https://arbital.com/p/-3gd) with these exercises.


Uncountable sample spaces are way too large

If the sample space $\Omega$ is [uncountable](https://arbital.com/p/2w0), then in general we can't even define a probability distribution over $\Omega$ in the same way we defined
[probability distributions](https://arbital.com/p/3tb) over countable sample spaces, i.e. by just assigning numbers to each point in the sample space.  Any function $f: \Omega \to [https://arbital.com/p/0,1](https://arbital.com/p/0,1)$ with $\sum_{\omega \in \Omega} f(\omega) = 1$ can only assign positive values to at most countably many elements of $\Omega$. But this means we can't, for example, talk about a
[uniform distribution](https://arbital.com/p/uniform_distribution) over the interval $[https://arbital.com/p/0,2](https://arbital.com/p/0,2)$, which intuitively should assign equal probability to everywhere in the interval.


Linear algebra

The study of [linear transformations](https://arbital.com/p/linear_transformation) and [vector spaces](https://arbital.com/p/vector_space).


Bit (of data): Examples

In the game "20 questions", one player (the "leader") thinks of a concept, and the other players ask 20 yes-or-no questions in attempts to guess that concept. In this game, the players extract 20 bits of data from the leader, which (if the questions are selected wisely) is enough data to single out one concept from a set of roughly one million concepts (1048576, to be precise).

Three bits of data are required to specify one of the hands of one of your biological grandparents: One bit for "maternal" or "paternal", one bit for "male" or "female," one bit for "left" or "right."

The president of the United States of America is selected from a set of about 320,000,000 people. This selection encodes roughly 28 bits of data. Where do those bits come from? Between 1/4 and 1/2 of the population is eligible for the presidency, so 1-2 bits come from constraints on who is allowed to be president. Another ~1 bit comes from the voters in the presidential election (when one candidate is selected from a set of two finalists). 2-4 more bits come from the primary elections (when the two finalists are selected from a set of ~4-32). However, most of the bits that go into choosing the president — 21 or more of them — come from self-selection and private politicking.


Fractional bits

It takes $\log_2(8) = 3$ [bits](https://arbital.com/p/3p0) of data to [carry](https://arbital.com/p/3y1) one message from a set of 8 possible messages. Similarly, it takes $\log_2(1024) = 10$ bits to carry one message from a set of 1024 possibilities. How many bits does it take to carry one message from a set of 3 possibilities? By the definition of "bit," the answers is $\log_2(3) \approx 1.58.$ What does this mean? That it takes about one and a half yes-or-no questions to single out one thing from a set of three? What is "half of a yes-or-no question"?

Fractional bits can be interpreted as telling about the [expected](https://arbital.com/p/4b5) cost of transmitting information: If you want to single out one thing from a set of 3 using bits (in, e.g., the [https://arbital.com/p/+3tz](https://arbital.com/p/+3tz) thought experiment) then you'll have to purchase two bits, but sometimes you'll be able to sell one of them back, which means you can push your expected cost lower than 2. How low can you push it? The lower bound is $\log_2(3),$ which is the minimum expected cost of adding a [3-message](https://arbital.com/p/3v9) to a long message when encoding your message as cleverly as possible. For more on this interpretation, see [https://arbital.com/p/+3v2](https://arbital.com/p/+3v2).

Fractional bits can also be interpreted as conversion rates between types of information: "A 3-message carries about 1.58 bits" can be interpreted as "one [https://arbital.com/p/3ww](https://arbital.com/p/3ww) is worth about 1.58 bits." To understand this, see [https://arbital.com/p/427](https://arbital.com/p/427), or [How many bits is a trit?](https://arbital.com/p/)

Fractional units of data can also be interpreted as a measure of how much we're using the digits allocated to encoding a number. For example, working with fractional [decits](https://arbital.com/p/3wn) instead of fractional bits, it only takes about 2.7 decits carry a [500-message](https://arbital.com/p/3v9), despite the fact that the number 500 clearly requires 3 decimal digits to write down. What's going on here? Well, we could declare that all numbers that start with a number $n \ge 5$ will be interpreted as if they start with the number $n - 5.$ Then we have two ways of representing each number (for example, 132 can be represented as both 132 and 632). Thus, if we have 3 decits but we only need to encode a 500-message, we have one bit to spare: We can encode one extra bit in our message according to whether we use the low representation or the high representation of the intended number. Thus, the amount of data it takes to communicate a 500-message is one bit lower than the amount of data it takes to encode a 1000-message — for a total cost of 3 decits minus one bit, which comes out to about 2.70 decits (or just short of 9 bits). For more on this interpretation, see [https://arbital.com/p/+44l](https://arbital.com/p/+44l).


GalCom

summary(Rules):

1. It costs 1 galcoin per bit to reserve on-peak bits in advance. Galcoins are very expensive.
2. You can reprogram your machine on Vega in advance, using cheap, off-peak bits. Don't worry about the cost of those.
3. You can resell reserved bits that you don't use (including fractional bits).
4. Any decrease in the expected cost of transmitting a message, no matter how small, is worthwhile.

_The GalCom hypothetical is a thought experiment where sending a bit of information is a clearly defined and very expensive action, which makes it useful for understanding various concepts in [https://arbital.com/p/-3qq](https://arbital.com/p/-3qq) and [https://arbital.com/p/-1bv](https://arbital.com/p/-1bv)._

The year is 21026. Humanity has become a flourishing inter-stellar civilization. Capitalism never died, and stock markets are thriving in various star systems. Due to the light-speed limitations, the markets are out of sync, and there's a lot of money to be made by setting up an automated trader in one star system, moving to another star system, and sending information about the stock market to your automated trader the instant that that information becomes available.

You set up your own automated trader in the Vega star system, and you currently make a living by transmitting market data from the Deneb star system. To transmit that data, you use the deep-space Galactic Communications network known as "GalCom."

You have a lot of data to transmit, and sending information on GalCom isn't cheap. Fortunately, it _is_ efficient: GalCom is a highly optimized network used by trillions of citizens to send and receive pulses of light across the cosmos. To send information via GalCom, you purchase [bits](https://arbital.com/p/3p0). For each bit you purchase, you are allowed to control a single pulse in the GalCom signal, making it either be present (representing a 1) or absent (representing a 0).

It's relatively cheap to buy bits on non-peak hours (to, e.g., reprogram your machine on Vega), but it's very expensive to reserve bits during peak hours (i.e., just after a juicy earnings report is published). You make your money by being among the very first people to send information about the stock market, so you have to reserve those very expensive precisely-timed bits in advance. This means that you have to know what messages you might want to send in advance, and reserve enough bits for the longest possible message that you might send. Fortunately, there's a secondary market on peak-hour bits, so if you end up reserving 5 bits and then only using 3 of them, you can resell the last two bits.

For example, say you know that you're going to send one of the following messages: `"buy"`, `"sell"`, or `"hold"`. GalCom would require you to purchase 4 bytes of information (enough to transmit four letters), and then if you actually end up transmitting `"buy"` you can sell the last byte back (because you only used 3 of the 4).

Of course, you can send that message using much less than 4 bytes, if you're clever, both by developing efficient [encoding schemes](https://arbital.com/p/) and by [accounting for the fact that some outcomes are more likely than others](https://arbital.com/p/). Your profit margins depend on you finding ways to transmit information as efficiently as possible. To do that, you can use [https://arbital.com/p/-3qq](https://arbital.com/p/-3qq).

In the GalCom thought experiment, you live in the future, and make your money by living in the Deneb star system and sending information to an automated stock trader you own in the Vega star system. Sending interstellar information is possible, but very expensive — and you have a lot of data to send. If you're clever, you can reprogram the computer on Vega in the low-cost off-hours, but to send the key market data (at the critical moment) you have to reserve high-cost precisely-timed bits far in advance. When reserving those bits you have to pick a fixed set of messages and buy enough bits to cover the longest message that you might want to send. If you end up sending a shorter message you can sell some of the bits back. You send a very high volume of data, so small decreases in the average cost of sending a bit are worth a lot of effort.