Multivariate random variable
In probability, and statistics, a multivariate random variable or random vector is a list of mathematical variables each of whose value is unknown, either because the value has not yet occurred or because there is imperfect knowledge of its value. The individual variables in a random vector are grouped together because they are all part of a single mathematical system — often they represent different properties of an individual statistical unit. For example, while a given person has a specific age, height and weight, the representation of these features of "an unspecified person" from within a group would  be a random vector. Normally each element of a random vector is a real number. Random vectors are often used as the underlying implementation of various types of aggregate random variables, e.g. a random matrix, random tree, random sequence, stochastic process, etc. More formally, a multivariate random variable is a column vector formula_1 (or its transpose, which is a row vector) whose components are scalar-valued random variables on the same probability space as each other, formula_2, where formula_3 is the sample space, formula_4 is the sigma-algebra (the collection of all events), and formula_5 is the probability measure (a  function returning each event's probability). Probability distribution. Every random vector gives rise to a probability measure on formula_6 with the Borel algebra as the underlying sigma-algebra. This measure is also known as the joint probability distribution, the joint distribution, or the multivariate distribution of the random vector. The distributions of each of the component random variables formula_7 are called marginal distributions. The conditional probability distribution of formula_7 given formula_9 is the probability distribution of formula_7 when formula_9 is known to be a particular value. The cumulative distribution function formula_12 of a random vector formula_13 is defined as where  formula_14. Operations on random vectors. Random vectors can be subjected to the same kinds of algebraic operations as can non-random vectors: addition, subtraction, multiplication by a scalar, and the taking of inner products. Affine transformations. Similarly, a new random vector formula_15 can be defined by applying an affine transformation formula_16 to a random vector formula_17: If formula_19 is an invertible matrix and formula_24 has a probability density function formula_25, then the probability density of formula_15 is Invertible mappings. More generally we can study invertible mappings of random vectors. Let formula_28 be a one-to-one mapping from an open  subset formula_29 of formula_6 onto a subset formula_31 of formula_6, let formula_28 have continuous partial derivatives in formula_29 and let the Jacobian determinant of formula_28 be zero at no point of formula_29. Assume that the real random vector formula_17 has a probability density function formula_38 and satisfies formula_39. Then the random vector formula_40 is of probability density where formula_42 denotes the indicator function and set formula_43 denotes support of formula_15. Expected value. The expected value or mean of a random vector formula_17 is a fixed vector formula_46 whose elements are the expected values of the respective random variables. Covariance and  cross-covariance. Definitions. The covariance matrix (also called second central moment or variance-covariance matrix) of an formula_47 random vector is an formula_48 matrix whose ("i,j") element is the covariance between the "i" and the "j" random variables. The covariance matrix is the expected value, element by element, of the formula_48 matrix computed as formula_50, where the superscript T refers to the transpose of the indicated vector: By extension, the cross-covariance matrix between two random vectors formula_17 and formula_15 (formula_17 having formula_54 elements and formula_15 having formula_56 elements) is the formula_57 matrix where again the matrix expectation is  taken element-by-element in the matrix. Here the ("i,j") element is the covariance between the "i" element of formula_17 and the "j" element of formula_15. Properties. The covariance matrix is a symmetric matrix, i.e. The covariance matrix is a positive semidefinite matrix, i.e. The cross-covariance matrix formula_62 is simply the transpose of the matrix formula_63, i.e. Uncorrelatedness. Two random vectors formula_65 and formula_66 are called uncorrelated if They are uncorrelated if and only if their cross-covariance matrix formula_68 is zero. Correlation and cross-correlation. Definitions. The correlation matrix (also called second moment) of an formula_47 random  vector is an formula_48 matrix whose ("i,j") element is the correlation between the "i" and the "j" random variables. The correlation matrix is the expected value, element by element, of the formula_48 matrix computed as formula_72, where the superscript T refers to the transpose of the indicated vector: By extension, the cross-correlation matrix between two random vectors formula_17 and formula_15 (formula_17 having formula_54 elements and formula_15 having formula_56 elements) is the formula_57 matrix Properties. The correlation matrix is related to the covariance matrix by Similarly for the cross-correlation matrix and the cross-covariance matrix: Orthogonality. Two random  vectors of the same size formula_82 and formula_66 are called orthogonal if Independence. Two random vectors formula_17 and formula_15 are called independent if for all formula_87 and formula_88 where formula_90 and formula_91 denote the cumulative distribution functions of formula_17 and formula_15 andformula_94 denotes their joint cumulative distribution function. Independence of formula_17 and formula_15 is often denoted by formula_97. Written component-wise, formula_17 and formula_15 are called independent if for all formula_100 Characteristic function. The characteristic function of a random vector formula_102 with formula_103 components is a function formula_104 that maps every vector formula_105 to a complex number. It is defined  by Further properties. Expectation of a quadratic form. One can take the expectation of a quadratic form in the random vector formula_17 as follows: where formula_109 is the covariance matrix of formula_17 and formula_111 refers to the trace of a matrix — that is, to the sum of the elements on its main diagonal (from upper left to lower right). Since the quadratic form is a scalar, so is its expectation. Proof: Let formula_112 be an formula_113 random vector with formula_114 and formula_115 and let formula_116 be an formula_117 non-stochastic matrix. Then based on the formula for the covariance, if  we denote formula_118 and formula_119, we see that: Hence which leaves us to show that This is true based on the fact that one can cyclically permute matrices when taking a trace without changing the end result (e.g.: formula_123). We see that And since is a scalar, then trivially. Using the permutation we get: and by plugging this into the original formula we get: Expectation of the product of two different quadratic forms. One can take the expectation of the product of two different quadratic forms in a zero-mean Gaussian random vector formula_17 as follows: where again formula_109 is  the covariance matrix of formula_17. Again, since both quadratic forms are scalars and hence their product is a scalar, the expectation of their product is also a scalar. Applications. Portfolio theory. In portfolio theory in finance, an objective often is to choose a portfolio of risky assets such that the distribution of the random portfolio return has desirable properties. For example, one might want to choose the portfolio return having the lowest variance for a given expected value. Here the random vector is the vector formula_133 of random returns on the individual assets, and the portfolio return "p" (a random  scalar) is the inner product of the vector of random returns with a vector "w" of portfolio weights — the fractions of the portfolio placed in the respective assets. Since "p" = "w"formula_133, the expected value of the portfolio return is "w"E(formula_133) and the variance of the portfolio return can be shown to be "w"C"w", where C is the covariance matrix of formula_133. Regression theory. In linear regression theory, we have data on "n" observations on a dependent variable "y" and "n" observations on each of "k" independent variables "x". The observations on the dependent variable are stacked into a column  vector "y"; the observations on each independent variable are also stacked into column vectors, and these latter column vectors are combined into a design matrix "X" (not denoting a random vector in this context) of observations on the independent variables. Then the following regression equation is postulated as a description of the process that generated the data: where β is a postulated fixed but unknown vector of "k" response coefficients, and "e" is an unknown random vector reflecting random influences on the dependent variable. By some chosen technique such as ordinary least squares, a vector formula_138 is chosen as an  estimate of β, and the estimate of the vector "e", denoted formula_139, is computed as Then the statistician must analyze the properties of formula_138 and formula_139, which are viewed as random vectors since a randomly different selection of "n" cases to observe would have resulted in different values for them. Vector time series. The evolution of a "k"×1 random vector formula_17 through time can be modelled as a vector autoregression (VAR) as follows: where the "i"-periods-back vector observation formula_145 is called the "i"-th lag of formula_17, "c" is a "k" × 1 vector of constants (intercepts), "A" is a  time-invariant "k" × "k" matrix and formula_147 is a "k" × 1 random vector of error terms.