Independence ( probability theory ) - wikipedia Independence ( probability theory ) Part of a series on Statistics Probability theory Probability axioms Probability space Sample space Elementary event Event Random variable Probability measure Complementary event Joint probability Marginal probability Conditional probability Independence Conditional independence Law of total probability Law of large numbers Bayes ' theorem Boole 's inequality Venn diagram Tree diagram In probability theory, two events are independent, statistically independent, or stochastically independent if the occurrence of one does not affect the probability of occurrence of the other. Similarly, two random variables are independent if the realization of one does not affect the probability distribution of the other. The concept of independence extends to dealing with collections of more than two events or random variables, in which case the events are pairwise independent if each pair are independent of each other, and the events are mutually independent if each event is independent of each other combination of events. Contents 1 Definition 1.1 For events 1.1. 1 Two events 1.1. 2 More than two events 1.2 For random variables 1.2. 1 Two random variables 1.2. 2 More than two random variables 1.2. 3 Conditional independence 1.3 Independent σ - algebras 2 Properties 2.1 Self - independence 2.2 Expectation and covariance 2.3 Characteristic function 3 Examples 3.1 Rolling dice 3.2 Drawing cards 3.3 Pairwise and mutual independence 3.4 Mutual independence 4 See also 5 References Definition For events Two events Two events A and B are independent ( often written as A ⊥ B ( displaystyle A perp B ) or A ⊥ ⊥ B ( displaystyle A perp ! ! ! perp B ) ) if their joint probability equals the product of their probabilities : P ( A ∩ B ) = P ( A ) P ( B ) ( displaystyle mathrm ( P ) ( A cap B ) = mathrm ( P ) ( A ) mathrm ( P ) ( B ) ). Why this defines independence is made clear by rewriting with conditional probabilities : P ( A ∩ B ) = P ( A ) P ( B ) ⇔ P ( A ) = P ( A ∩ B ) P ( B ) = P ( A ∣ B ) ( displaystyle mathrm ( P ) ( A cap B ) = mathrm ( P ) ( A ) mathrm ( P ) ( B ) Leftrightarrow mathrm ( P ) ( A ) = ( \frac ( mathrm ( P ) ( A cap B ) ) ( mathrm ( P ) ( B ) ) ) = mathrm ( P ) ( A mid B ) ). and similarly P ( A ∩ B ) = P ( A ) P ( B ) ⇔ P ( B ) = P ( B ∣ A ) ( displaystyle mathrm ( P ) ( A cap B ) = mathrm ( P ) ( A ) mathrm ( P ) ( B ) Leftrightarrow mathrm ( P ) ( B ) = mathrm ( P ) ( B mid A ) ). Thus, the occurrence of B does not affect the probability of A, and vice versa. Although the derived expressions may seem more intuitive, they are not the preferred definition, as the conditional probabilities may be undefined if P ( A ) or P ( B ) are 0. Furthermore, the preferred definition makes clear by symmetry that when A is independent of B, B is also independent of A. More than two events A finite set of events ( A i ) i = 1 n ( displaystyle ( A_ ( i ) ) _ ( i = 1 ) ^ ( n ) ) is pairwise independent if every pair of events is independent -- that is, if and only if for all distinct pairs of indices m, k, P ( A m ∩ A k ) = P ( A m ) P ( A k ). ( displaystyle mathrm ( P ) ( A_ ( m ) cap A_ ( k ) ) = mathrm ( P ) ( A_ ( m ) ) mathrm ( P ) ( A_ ( k ) ). ) A finite set of events is mutually independent if every event is independent of any intersection of the other events -- that is, if and only if for every k - element subset of ( A i ) i = 1 n ( displaystyle ( A_ ( i ) ) _ ( i = 1 ) ^ ( n ) ), P ( ⋂ i = 1 k A i ) = ∏ i = 1 k P ( A i ). ( displaystyle mathrm ( P ) left ( \bigcap _ ( i = 1 ) ^ ( k ) A_ ( i ) \right ) = prod _ ( i = 1 ) ^ ( k ) mathrm ( P ) ( A_ ( i ) ). ) This is called the multiplication rule for independent events. Note that it is not a single condition involving only the product of all the probabilities of all single events ( see below for a counterexample ) ; it must hold true for all subsets of events. For more than two events, a mutually independent set of events is ( by definition ) pairwise independent ; but the converse is not necessarily true ( see below for a counterexample ). For random variables Two random variables Two random variables X and Y are independent if and only if ( iff ) the elements of the π - system generated by them are independent ; that is to say, for every a and b, the events ( X ≤ a ) and ( Y ≤ b ) are independent events ( as defined above ). That is, X and Y with cumulative distribution functions F X ( x ) ( displaystyle F_ ( X ) ( x ) ) and F Y ( y ) ( displaystyle F_ ( Y ) ( y ) ), and probability densities f X ( x ) ( displaystyle f_ ( X ) ( x ) ) and f Y ( y ) ( displaystyle f_ ( Y ) ( y ) ), are independent iff the combined random variable ( X, Y ) has a joint cumulative distribution function F X, Y ( x, y ) = F X ( x ) F Y ( y ), ( displaystyle F_ ( X, Y ) ( x, y ) = F_ ( X ) ( x ) F_ ( Y ) ( y ), ) or equivalently, if the joint density exists, f X, Y ( x, y ) = f X ( x ) f Y ( y ). ( displaystyle f_ ( X, Y ) ( x, y ) = f_ ( X ) ( x ) f_ ( Y ) ( y ). ) More than two random variables A set of random variables is pairwise independent if and only if every pair of random variables is independent. Even if the set of random variables is pairwise independent, it is not necessarily mutually independent as defined next. A set of random variables is mutually independent if and only if for any finite subset X 1,..., X n ( displaystyle X_ ( 1 ), ldots, X_ ( n ) ) and any finite sequence of numbers a 1,..., a n ( displaystyle a_ ( 1 ), ldots, a_ ( n ) ), the events ( X 1 ≤ a 1 ),..., ( X n ≤ a n ) ( displaystyle ( X_ ( 1 ) leq a_ ( 1 ) ), ldots, ( X_ ( n ) leq a_ ( n ) ) ) are mutually independent events ( as defined above ). The measure - theoretically inclined may prefer to substitute events ( X ∈ A ) for events ( X ≤ a ) in the above definition, where A is any Borel set. That definition is exactly equivalent to the one above when the values of the random variables are real numbers. It has the advantage of working also for complex - valued random variables or for random variables taking values in any measurable space ( which includes topological spaces endowed by appropriate σ - algebras ). Conditional independence Main article : Conditional independence Intuitively, two random variables X and Y are conditionally independent given Z if, once Z is known, the value of Y does not add any additional information about X. For instance, two measurements X and Y of the same underlying quantity Z are not independent, but they are conditionally independent given Z ( unless the errors in the two measurements are somehow connected ). The formal definition of conditional independence is based on the idea of conditional distributions. If X, Y, and Z are discrete random variables, then we define X and Y to be conditionally independent given Z if P ( X ≤ x, Y ≤ y Z = z ) = P ( X ≤ x Z = z ) ⋅ P ( Y ≤ y Z = z ) ( displaystyle mathrm ( P ) ( X leq x, Y leq y ; ; Z = z ) = mathrm ( P ) ( X leq x ; ; Z = z ) cdot mathrm ( P ) ( Y leq y ; ; Z = z ) ) for all x, y and z such that P ( Z = z ) > 0. On the other hand, if the random variables are continuous and have a joint probability density function p, then X and Y are conditionally independent given Z if p X Y Z ( x, y z ) = p X Z ( x z ) ⋅ p Y Z ( y z ) ( displaystyle p_ ( XY Z ) ( x, y z ) = p_ ( X Z ) ( x z ) cdot p_ ( Y Z ) ( y z ) ) for all real numbers x, y and z such that p ( z ) > 0. If discrete X and Y are conditionally independent given Z, then P ( X = x Y = y, Z = z ) = P ( X = x Z = z ) ( displaystyle mathrm ( P ) ( X = x Y = y, Z = z ) = mathrm ( P ) ( X = x Z = z ) ) for any x, y and z with P ( Z = z ) > 0. That is, the conditional distribution for X given Y and Z is the same as that given Z alone. A similar equation holds for the conditional probability density functions in the continuous case. Independence can be seen as a special kind of conditional independence, since probability can be seen as a kind of conditional probability given no events. Independent σ - algebras The definitions above are both generalized by the following definition of independence for σ - algebras. Let ( Ω, Σ, P ) ( displaystyle ( Omega, Sigma, mathrm ( P ) ) ) be a probability space and let A ( displaystyle ( mathcal ( A ) ) ) and B ( displaystyle ( mathcal ( B ) ) ) be two sub-σ - algebras of Σ ( displaystyle Sigma ). A ( displaystyle ( mathcal ( A ) ) ) and B ( displaystyle ( mathcal ( B ) ) ) are said to be independent if, whenever A ∈ A ( displaystyle A in ( mathcal ( A ) ) ) and B ∈ B ( displaystyle B in ( mathcal ( B ) ) ), P ( A ∩ B ) = P ( A ) P ( B ). ( displaystyle mathrm ( P ) ( A cap B ) = mathrm ( P ) ( A ) mathrm ( P ) ( B ). ) Likewise, a finite family of σ - algebras ( τ i ) i ∈ I ( displaystyle ( \tau _ ( i ) ) _ ( i in I ) ), where I ( displaystyle I ) is an index set, is said to be independent if and only if ∀ ( A i ) i ∈ I ∈ ∏ i ∈ I τ i : P ( ⋂ i ∈ I A i ) = ∏ i ∈ I P ( A i ) ( displaystyle \forall left ( A_ ( i ) \right ) _ ( i in I ) in prod \nolimits _ ( i in I ) \tau _ ( i ) :mathrm ( P ) left ( \bigcap \nolimits _ ( i in I ) A_ ( i ) \right ) = prod \nolimits _ ( i in I ) mathrm ( P ) left ( A_ ( i ) \right ) ) and an infinite family of σ - algebras is said to be independent if all its finite subfamilies are independent. The new definition relates to the previous ones very directly : Two events are independent ( in the old sense ) if and only if the σ - algebras that they generate are independent ( in the new sense ). The σ - algebra generated by an event E ∈ Σ ( displaystyle E in Sigma ) is, by definition, σ ( ( E ) ) = ( ∅, E, Ω ∖ E, Ω ). ( displaystyle sigma ( ( E ) ) = ( emptyset, E, Omega setminus E, Omega ). ) Two random variables X and Y defined over Ω are independent ( in the old sense ) if and only if the σ - algebras that they generate are independent ( in the new sense ). The σ - algebra generated by a random variable X taking values in some measurable space S consists, by definition, of all subsets of Ω of the form X ( U ), where U is any measurable subset of S. Using this definition, it is easy to show that if X and Y are random variables and Y is constant, then X and Y are independent, since the σ - algebra generated by a constant random variable is the trivial σ - algebra ( ∅, Ω ). Probability zero events can not affect independence so independence also holds if Y is only Pr - almost surely constant. Properties Self - independence Note that an event is independent of itself if and only if P ( A ) = P ( A ∩ A ) = P ( A ) ⋅ P ( A ) ⇔ P ( A ) = 0 or 1 ( displaystyle mathrm ( P ) ( A ) = mathrm ( P ) ( A cap A ) = mathrm ( P ) ( A ) cdot mathrm ( P ) ( A ) Leftrightarrow mathrm ( P ) ( A ) = 0 ( \text ( or ) ) 1 ). Thus an event is independent of itself if and only if it almost surely occurs or its complement almost surely occurs. Expectation and covariance If X and Y are independent, then the expectation operator E has the property E ( X Y ) = E ( X ) E ( Y ), ( displaystyle E ( XY ) = E ( X ) E ( Y ), ) and the covariance cov ( X, Y ) is zero, since we have cov ( X, Y ) = E ( X Y ) − E ( X ) E ( Y ). ( displaystyle ( \text ( cov ) ) ( X, Y ) = E ( XY ) - E ( X ) E ( Y ). ) ( The converse of these, i.e. the proposition that if two random variables have a covariance of 0 they must be independent, is not true. See uncorrelated. ) Characteristic function Two random variables X and Y are independent if and only if the characteristic function of the random vector ( X, Y ) satisfies φ ( X, Y ) ( t, s ) = φ X ( t ) ⋅ φ Y ( s ). ( displaystyle \varphi _ ( ( X, Y ) ) ( t, s ) = \varphi _ ( X ) ( t ) cdot \varphi _ ( Y ) ( s ). ) In particular the characteristic function of their sum is the product of their marginal characteristic functions : φ X + Y ( t ) = φ X ( t ) ⋅ φ Y ( t ), ( displaystyle \varphi _ ( X + Y ) ( t ) = \varphi _ ( X ) ( t ) cdot \varphi _ ( Y ) ( t ), ) though the reverse implication is not true. Random variables that satisfy the latter condition are called subindependent. Examples Rolling dice The event of getting a 6 the first time a die is rolled and the event of getting a 6 the second time are independent. By contrast, the event of getting a 6 the first time a die is rolled and the event that the sum of the numbers seen on the first and second trials is 8 are not independent. Drawing cards If two cards are drawn with replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are independent. By contrast, if two cards are drawn without replacement from a deck of cards, the event of drawing a red card on the first trial and that of drawing a red card on the second trial are not independent, because a deck that has had a red card removed has proportionately fewer red cards. Pairwise and mutual independence Pairwise independent, but not mutually independent, events. Mutually independent events. Consider the two probability spaces shown. In both cases, P ( A ) = P ( B ) = 1 / 2 and P ( C ) = 1 / 4. The random variables in the first space are pairwise independent because P ( A B ) = P ( A C ) = 1 / 2 = P ( A ), P ( B A ) = P ( B C ) = 1 / 2 = P ( B ), and P ( C A ) = P ( C B ) = 1 / 4 = P ( C ) ; but the three random variables are not mutually independent. The random variables in the second space are both pairwise independent and mutually independent. To illustrate the difference, consider conditioning on two events. In the pairwise independent case, although any one event is independent of each of the other two individually, it is not independent of the intersection of the other two : P ( A B C ) = 4 40 4 40 + 1 40 = 4 5 ≠ P ( A ) ( displaystyle mathrm ( P ) ( A BC ) = ( \frac ( \frac ( 4 ) ( 40 ) ) ( ( \frac ( 4 ) ( 40 ) ) + ( \frac ( 1 ) ( 40 ) ) ) ) = ( \tfrac ( 4 ) ( 5 ) ) \neq mathrm ( P ) ( A ) ) P ( B A C ) = 4 40 4 40 + 1 40 = 4 5 ≠ P ( B ) ( displaystyle mathrm ( P ) ( B AC ) = ( \frac ( \frac ( 4 ) ( 40 ) ) ( ( \frac ( 4 ) ( 40 ) ) + ( \frac ( 1 ) ( 40 ) ) ) ) = ( \tfrac ( 4 ) ( 5 ) ) \neq mathrm ( P ) ( B ) ) P ( C A B ) = 4 40 4 40 + 6 40 = 2 5 ≠ P ( C ) ( displaystyle mathrm ( P ) ( C AB ) = ( \frac ( \frac ( 4 ) ( 40 ) ) ( ( \frac ( 4 ) ( 40 ) ) + ( \frac ( 6 ) ( 40 ) ) ) ) = ( \tfrac ( 2 ) ( 5 ) ) \neq mathrm ( P ) ( C ) ) In the mutually independent case, however, P ( A B C ) = 1 16 1 16 + 1 16 = 1 2 = P ( A ) ( displaystyle mathrm ( P ) ( A BC ) = ( \frac ( \frac ( 1 ) ( 16 ) ) ( ( \frac ( 1 ) ( 16 ) ) + ( \frac ( 1 ) ( 16 ) ) ) ) = ( \tfrac ( 1 ) ( 2 ) ) = mathrm ( P ) ( A ) ) P ( B A C ) = 1 16 1 16 + 1 16 = 1 2 = P ( B ) ( displaystyle mathrm ( P ) ( B AC ) = ( \frac ( \frac ( 1 ) ( 16 ) ) ( ( \frac ( 1 ) ( 16 ) ) + ( \frac ( 1 ) ( 16 ) ) ) ) = ( \tfrac ( 1 ) ( 2 ) ) = mathrm ( P ) ( B ) ) P ( C A B ) = 1 16 1 16 + 3 16 = 1 4 = P ( C ) ( displaystyle mathrm ( P ) ( C AB ) = ( \frac ( \frac ( 1 ) ( 16 ) ) ( ( \frac ( 1 ) ( 16 ) ) + ( \frac ( 3 ) ( 16 ) ) ) ) = ( \tfrac ( 1 ) ( 4 ) ) = mathrm ( P ) ( C ) ) Mutual independence It is possible to create a three - event example in which P ( A ∩ B ∩ C ) = P ( A ) P ( B ) P ( C ), ( displaystyle mathrm ( P ) ( A cap B cap C ) = mathrm ( P ) ( A ) mathrm ( P ) ( B ) mathrm ( P ) ( C ), ) and yet no two of the three events are pairwise independent ( and hence the set of events are not mutually independent ). This example shows that mutual independence involves requirements on the products of probabilities of all combinations of events, not just the single events as in this example. See also Copula ( statistics ) Independent and identically distributed random variables Mutually exclusive events Subindependence Linear dependence between random variables Conditional independence Normally distributed and uncorrelated does not imply independent Mean dependence References Jump up ^ Russell, Stuart ; Norvig, Peter ( 2002 ). Artificial Intelligence : A Modern Approach. Prentice Hall. p. 478. ISBN 0 - 13 - 790395 - 2. ^ Jump up to : Feller, W ( 1971 ). `` Stochastic Independence ''. An Introduction to Probability Theory and Its Applications. Wiley. Jump up ^ George, Glyn, `` Testing for the independence of three events, '' Mathematical Gazette 88, November 2004, 568. PDF Retrieved from `` https://en.wikipedia.org/w/index.php?title=Independence_(probability_theory)&oldid=799275720 '' Categories : Experiment ( probability theory ) Independence ( probability theory ) Talk Contents About Wikipedia Català Deutsch Español Esperanto Euskara فارسی Français 한국어 Italiano עברית Bahasa Melayu Nederlands 日本 語 Polski Português Русский Simple English Slovenščina Basa Sunda Suomi Svenska ไทย Türkçe Українська اردو Tiếng Việt 中文 Edit links This page was last edited on 6 September 2017, at 18 : 43. Text is available under the Creative Commons Attribution - ShareAlike License ; additional terms may apply. By using this site, you agree to the Terms of Use and Privacy Policy. Wikipedia ® is a registered trademark of the Wikimedia Foundation, Inc., a non-profit organization. About Wikipedia

Question: when are two events said to be independent
Answer: if the occurrence of one does not affect the probability of occurrence of the other; if the occurrence of one does not affect the probability of occurrence of the other; if the occurrence of one does not affect the probability of occurrence of the other.