# A Note on Random Variables

$\newcommand{\triple}{(\Omega, \mathcal{F}, \mathbf{P})}$$\newcommand{\P}{\mathbf{P}}$ This note on random variables follows as a result of confusing notation in several math textbooks. I'll explain random variables (in measure theoretic terms) as verbosely as I can, and then prove some results. This article assumes that the reader is familiar with probability triples $\triple$, as well as a basic idea of what random variables are, in non-measure theory terms.

## 1\. Random Variable Prerequisites

We start with defining measurable spaces and measurable functions 

> **Definition 1.1** : A Measurable space $(X,\Sigma)$ consists of a set $X$ and a $\sigma$-algebra $\Sigma$ defined on $X$.
> 
> **Definition 1.2** : A Generated $\sigma$-algebra is the smallest $\sigma$-algebra containing a specified collection of sets. That is, if $A$ is a set of subsets of $X$, $\sigma(A)$ is the smallest sigma-algebra such that $A \subseteq \sigma(A)$. 
> 
> **Definition 1.3** : A Measurable Function $f: (X,\Sigma) \to (Y,\Gamma)$ between two measurable spaces is a function such that for every $E \in \Gamma$, $\\{x \in X\ |\ f(x) \in E\\} \in \Sigma$.

Since the definition $\\{x \in X\ |\ f(x) \in E\\}$ is used so commonly in the context of measurable functions, this has a special notation

> **Definition 1.4** : $$f^{-1}(E) := \\{x \in X\ |\ f(x) \in E\\}$$

NOTE: The above definition is confusing, but is unfortunately the norm when dealing with measurable functions. In the context of measurable functions, $f^{-1}$ does not refer to the inverse of $f$ (which is a function from $Y \to X$), but rather the set of preimages of all the elements contained in a set in the sigma algebra.

Measurable functions can also be defined in terms of the $\sigma$-algebra generated by a _function_ , rather than that of a set

> **Definition 1.5** : The $\sigma$-algebra generated by a function $f: (X,\Sigma) \to (Y, \Gamma)$ is the collection of all inverse images $f^{-1}(S),\ S \in \Gamma$. $$\sigma(f) := \\{ f^{-1}(S) : S \in \Gamma \\}$$

According to this definition, if $\sigma(f) \subseteq \Sigma$, then $f$ is a measurable function.

## 2\. Random Variables

Random variables are unfortunately, neither random nor variables. This is the first of many misnomers that we encounter in their study. 

> **Definition 2.1** : A Random Variable $X$ defined on a probability triple $\triple$ is a measurable function $X : (\Omega, \mathcal{F}) \to (\mathbb{R}, \mathcal{B}(\mathbb{R}))$

In it's simplest terms, A random variable is simply a function from $\Omega \to \mathbb{R}$, obeying some 'nice' rules which allow us to use probability measures with it. These nice rules would come into play a bit later, after we first see how random variables and probability measures go hand in hand.

Consider $$\begin{align}\Omega &= \\{1,2,3\\}\\\ \mathcal{F} &= 2^{\Omega} \\\ \mathbf{P}&:\mathcal{F} \to [0,1]\end{align}$$ such that $\mathbf{P}\\{1\\} = \mathbf{P}\\{2\\} = \mathbf{P}\\{3\\}$ (This is the discrete uniform probability space on $\\{1,2,3\\}$). Let our random variable $X: \Omega \to \mathbb{R}$ map $\\{i\\}$ to $i$, $i \in \\{1,2,3\\}$. A graphical depiction of this would look something like this:

Now, suppose we had to calculate the probability that the random variable $X$ would be less than or equal to $2.5$. The probability of this event occuring is given by $\P\\{\omega \in \Omega\ :\ X(\omega) \le 2.5\\}$. From the inverse notation we developed in $\S$1.4, We can also write this as $\P(X^{-1}((-\infty, 2.5]))$. From the graph, we clearly see that $1$ and $2$ are the only elements in $\Omega$ that would be in this set, hence $\P\\{1,2\\} = 2/3$. This is how random variables and probability measures go hand in hand.

Why then, do random variables need to be measurable functions? **Note that the probability measure is only defined for sets in $\mathcal{F}$, and if $X$ is not measurable, we cannot find the probability of certain events associated with $X$**. 

An example for this is to consider $$\begin{align}\Omega &= \\{1,2,3\\}\\\ \mathcal{F'} &= \\{\emptyset, \\{1\\}, \\{2,3\\}, \Omega\\} \\\ \mathbf{P}&:\mathcal{F'} \to [0,1]\end{align}$$ such that $\P\\{1\\} = 1/3$. Now consider $X': \Omega \to \mathbb{R}$ such that $X'(i) = i,\ i \in \Omega$. This is the same map as before. However, if we try to calculate the probability that $X$ is less than or equal to 2.5 now, **we find that $\P\\{1,2\\}$ is undefined, as $\\{1,2\\} \not\in \mathcal{F'}$**. Hence, $X'$ is not a random variable, as it is not measurable on $(\Omega, \mathcal{F'})$. More specifically, $\sigma(X) = 2^\Omega \not\subseteq \mathcal{F'}$, hence, $X'$ is not measurable

## 3\. Results on Random Variables

> **Claim 3.1** : If $X: \Omega \to \mathbb(R)$ is a random variable on $\triple$, then $X^{-1}(B) = A \implies X^{-1}(B^C) = A^C$. 

A simple (maybe even obvious) claim, the proof is by definition: $$\begin{align} X^{-1}(B) &= A = \\{\omega \in \Omega\ :\ X(\omega) \in B\\} \\\ \implies X^{-1}(B^C) &= \\{\omega \in \Omega\ :\ X(\omega) \not\in B\\} = A^C \end{align}$$

> **Claim 3.2** : If $X = \mathbf{1}_A$ is the indicator of some event $A \in \mathcal{F}$, then $X$ is a random variable

Proof: for all $B \in \mathcal{B}(\mathbb{R})$, we have $X(B)$ equal to any one of $A$ (if $B$ contains 1 and not 0), $A^C$ (if $B$ contains 0 and not 1), $\emptyset$ (if $B$ contains neither 0 nor 1) or $\Omega$ (if $B$ contains both 0 and 1). Hence, $X$ is a random variable.

The next two claims would be key to proving results about functions of random variables

> **Claim 3.3** : if $f: (\Omega_1, \mathcal{F}_1) \to (\Omega_2, \mathcal{F}_2)$ and $g: (\Omega_2, \mathcal{F}_2) \to (\Omega_3, \mathcal{F}_3)$ are two measurable functions, then $f \circ g : (\Omega_1, \mathcal{F}_1) \to (\Omega_3, \mathcal{F}_3)$ is also a measurable function

Proof: For all $B \in \mathcal{F}_3$, since $g$ is measurable, $g^{-1}(B) \in \mathcal{F}_2$. Since $f$ is measurable, $f^{-1}(g^{-1}(B)) \in \mathcal{F}_1$. Hence, $f\circ g$ is measurable.

> **Claim 3.4** : $f: (\Omega_1, \mathcal{F}_1) \to (\Omega_2, \sigma(C))$ is measurable if $A \in C \implies f^{-1}(A) \in \mathcal{F}_1$.

Proof: Note that $f^{-1}(\Omega_2 \setminus A) = \Omega_2 \setminus f^{-1}(A)$, and $f^{-1}(\cup_n A_n) = \cup_n f^{-1}(A_n)$. This, along with the fact that $\mathcal{F}_1$ is a $\sigma$-algebra proves that $\\{A : f^{-1}(A) \in \mathcal{F}_1\\}$ is a $\sigma$-algebra containing $C$. Since $\sigma(C)$ is the smallest $\sigma$-algbra containing C, $\sigma(C)$ would be a subset of the above $\sigma$-algebra, hence the claim is true.

This above claim ensures that we don't need to prove that every set of a $\sigma$-algebra has a preimage in the previous $\sigma$-algebra. Proving it for only the generating set is enough eg. for $\mathcal{B}(\mathbb{R})$, it's sufficient to show that only the open sets have a preimage, something that we'll use in the next proof.

> **Claim 3.5** : Every continuous function $f: \mathbb{R} \to \mathbb{R}$ is measurable.

Proof: from (3.4), it's sufficient to prove that for every open set $A$, $f^{-1}(A) \in \mathcal{B}(\mathbb{R})$. This follows from the continuity of $f$: $f$ is continuous iff $G$ is open implies that $f^{-1}(G)$ is also open. Hence, $f$ is measurable.

The above three claims give us the following very powerful result: **every continuous function of a random variable is also a random variable**. We can make a stronger claim, after proving the following claims as well:

> **Claim 3.6** : If $X$ and $Y$ are random variables on $\triple$, then $X+Y$ and $XY$ are random variables as well

Proof: This cute proof comes from Rosenthal. It's sufficient to prove that $X+Y$ is a random variable on the collection of sets $(-\infty, x)$, as the generated $\sigma$-algebra of this collection is $\mathcal{B}(\mathbb{R})$. Hence, consider the set $\\{\omega \in \Omega : X(\omega) + Y(\omega) < x\\}$. From the density theorem, we can find a rational number in $(X, x-Y)$ (I've dropped the $(\omega)$, as it's implicit here). hence, $$\\{X + Y < x\\} = \bigcup_{\text{r rational}} (\\{X \lt r\\} \cap \\{Y \lt x - r\\})$$ Since all the elements in the union belong to $\mathcal{F}$ and since $\mathcal{F}$ is a $\sigma$-algebra, $X+Y$ is a random variable.

XY is also a random variable, as $XY = [(X+Y)^2 - (X^2+Y^2)]/2$, and a sum/function of random variables is a random variable, from the previous claims.

We are now free to extend the claim that every continuous function of a random variable is a random variable, to piecewise continuity: **every piecewise continuous function of a random variable is also a random variable**. If $f$ is piecewise continuous, then $f(X) = f_1(X) \mathbf{1}_{I_1} + f_2(X) \mathbf{1}_{I_2} + \ldots + f_n(X) \mathbf{1}_{I_n}$, where $f_j(X)$ are random variables as $f_j$ is continuous, and $I_j$ are disjoint intervals. From claim (3.6), $f(X)$ is a linear sum of random variables, and hence is also a random variable.

## 4\. References

  1. Rosenthal, Jeffrey S. _A First Look at Rigorous Probability Theory_. World Scientific, 2006. _Open WorldCat_ , http://public.ebookcentral.proquest.com/choice/publicfullrecord.aspx?p=5227675
  2. Lebanon, Guy, editor. _Probability: The Analysis of Data ; Vol. 1._ 2012\. Available online at http://theanalysisofdata.com/probability/0_2.html
  3. Math StackExchange, Wikipedia, etc etc :)