# Statistics/Probability/Bayesian

Statistics
* Introduction
** What Is Statistics?
** Subjects in Modern Statistics
** Why Should I Learn Statistics?
** What Do I Need to Know to Learn Statistics?
* Different Types of Data
** Primary and Secondary Data
** Quantitative and Qualitative Data
* Methods of Data Collection
** Experiments
** Sample Surveys
** Observational Studies
* Data Analysis
** Data Cleaning
** Moving Average
* Summary Statistics
** Measures of center
*** Mean, Median, and Mode
*** Geometric Mean
*** Harmonic Mean
*** Relationships among Arithmetic, Geometric, and Harmonic Mean
*** Geometric Median
** Measures of dispersion
*** Range of the Data
*** Variance and Standard Deviation
*** Quartiles and Quartile Range
*** Quantiles
* Displaying Data
** Bar Charts
** Comparative Bar Charts
** Histograms
** Scatter Plots
** Box Plots
** Pie Charts
** Comparative Pie Charts
** Pictograms
** Line Graphs
** Frequency Polygon
* Probability
** Combinatorics
** Bernoulli Trials
** Introductory Bayesian Analysis
* Distributions
** Discrete Distributions
*** Uniform Distribution
*** Bernoulli Distribution
*** Binomial Distribution
*** Poisson Distribution
*** Geometric Distribution
*** Negative Binomial Distribution
*** Hypergeometric Distribution
** Continuous Distributions
*** Uniform Distribution
*** Exponential Distribution
*** Gamma Distribution
*** Normal Distribution
*** Chi-Square Distribution
*** Student-t Distribution
*** F Distribution
*** Beta Distribution
*** Weibull Distribution
* Testing Statistical Hypothesis
** Purpose of Statistical Tests
** Formalism Used
** Different Types of Tests
** z Test for a Single Mean
** z Test for Two Means
** t Test for a single mean
** t Test for Two Means
** paired t Test for comparing Means
** One-Way ANOVA F Test
** z Test for a Single Proportion
** z Test for Two Proportions
** Testing whether Proportion A Is Greater than Proportion B in Microsoft Excel
** Spearman's Rank Coefficient
** Pearson's Product Moment Correlation Coefficient
** Chi-Squared Tests
*** Chi-Squared Test for Multiple Proportions
*** Chi-Squared Test for Contingency
** Approximations of distributions
* Point Estimates (12:07, 28 March 2007 (UTC))
** Unbiasedness
** Measures of goodness
** UMVUE
** Completeness
** Sufficiency and Minimal Sufficiency
** Ancillarity
* Practice Problems
** Summary Statistics Problems
** Data-Display Problems
** Distributions Problems
** Data-Testing Problems
* Numerical Methods
** Basic Linear Algebra and Gram-Schmidt Orthogonalization
** Unconstrained Optimization
** Quantile Regression
** Numerical Comparison of Statistical Software
** Numerics in Excel
** Statistics/Numerical_Methods/Random Number Generation
* Time Series Analysis
* Multivariate Data Analysis
** Principal Component Analysis
** Factor Analysis for metrical data
** Factor Analysis for ordinal data
** Canonical Correlation Analysis
** Discriminant Analysis
* Analysis of Specific Datasets
** Analysis of Tuberculosis
* Appendix
** Authors
** Glossary
** Index
** Links
Bayesian analysis is the branch of statistics based on the idea that we have some knowledge in advance about the probabilities that we are interested in, so called a priori probabilities. This might be your degree of belief in a particular event, the results from previous studies, or a general agreed-upon starting value for a probability. The terminology "Bayesian" comes from the Bayesian rule or law, a law about conditional probabilities. The opposite of "Bayesian" is sometimes referred to as "Frequentist Statistics."

### Example

Consider a box with 3 coins, with probabilities of showing heads respectively 1/4, 1/2 and 3/4. We choose arbitrarily one of the coins. Hence we take 1/3 as the a priori probability $P(C_{1})$ of having chosen coin number 1. After 5 throws, in which X=4 times heads came up, it seems less likely that the coin is coin number 1. We calculate the a posteriori probability that the coin is coin number 1, as:
    ${\begin{aligned}P(C_{1}|X=4)&={\frac {P(X=4|C_{1})P(C_{1})}{P(X=4)}}\\&={\frac {P(X=4|C_{1})P(C_{1})}{P(X=4|C_{1})P(C_{1})+P(X=4|C_{2})P(C_{2})+P(X=4|C_{3})P(C_{3})}}\\&={\frac {{5 \choose 4}({\frac {1}{4}})^{4}{\frac {3}{4}}{\frac {1}{3}}}{{5 \choose 4}({\frac {1}{4}})^{4}{\frac {3}{4}}{\frac {1}{3}}+{5 \choose 4}({\frac {1}{2}})^{4}{\frac {1}{2}}{\frac {1}{3}}+{5 \choose 4}({\frac {3}{4}})^{4}{\frac {1}{4}}{\frac {1}{3}}}}\end{aligned}}$
In words:
    The probability that the Coin is the first Coin, given that we know heads came up 4 times... Is equal to the probability that heads came up 4 times given we know it's the first coin, times the probability that the coin is the first coin. All divided by the probability that heads comes up 4 times (ignoring which of the three Coins is chosen). The binomial coefficients cancel out as well as all denominators when expanding 1/2 to 2/4. This results in
       ${\frac {3}{3+32+81}}={\frac {3}{116}}$
In the same way we find:
    $P(C_{2}|X=4)={\frac {32}{3+32+81}}={\frac {32}{116}}$
and
    $P(C_{3}|X=4)={\frac {81}{3+32+81}}={\frac {81}{116}}$.
This shows us that after examining the outcome of the five throws, it is most likely we did choose coin number 3.
Actually for a given result the denominator does not matter, only the relative Probabilities $p(C_{i})=P(C_{i}|X=4)/P(X=4)$ When the result is 3 times heads the Probabilities change in favor of Coin 2 and further as the following table shows:
| Heads | $p(C_{1})$ | $p(C_{2})$ | $p(C_{3})$ |
| 5 | 1 | 32 | 243 |
| 4 | 3 | 32 | 81 |
| 3 | 9 | 32 | 27 |
| 2 | 27 | 32 | 9 |
| 1 | 81 | 32 | 3 |
| 0 | 243 | 32 | 1 |