User: list of common continuous prob distributions and their density functions
Assistant: Here is a list of common continuous probability distributions along with their probability density functions (pdf):

### 1. Uniform Distribution \( \text{Uniform}(a,b) \)
- **Support:** \( x \in [a,b] \)
- **PDF:** 
\[
f(x) = \frac{1}{b - a}, \quad a \leq x \leq b
\]

---

### 2. Normal (Gaussian) Distribution \( \mathcal{N}(\mu, \sigma^2) \)
- **Support:** \( x \in (-\infty, \infty) \)
- **PDF:** 
\[
f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right)
\]

---

### 3. Exponential Distribution \( \text{Exp}(\lambda) \)
- **Support:** \( x \geq 0 \)
- **PDF:** 
\[
f(x) = \lambda e^{-\lambda x}
\]

---

### 4. Gamma Distribution \( \Gamma(\alpha, \beta) \) (shape \(\alpha > 0\), rate \(\beta > 0\))
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\beta x}
\]

---

### 5. Beta Distribution \( \text{Beta}(\alpha, \beta) \)
- **Support:** \( x \in [0,1] \)
- **PDF:** 
\[
f(x) = \frac{x^{\alpha - 1} (1 - x)^{\beta - 1}}{B(\alpha, \beta)}
\]
where \( B(\alpha, \beta) = \frac{\Gamma(\alpha) \Gamma(\beta)}{\Gamma(\alpha + \beta)} \)

---

### 6. Chi-Square Distribution (\(\chi^2_k\)) — Special case of Gamma with \(\alpha = k/2, \beta = 1/2\)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{1}{2^{k/2} \Gamma(k/2)} x^{\frac{k}{2} - 1} e^{-x/2}
\]

---

### 7. Student's t-Distribution \( t_\nu \)
- **Support:** \( x \in (-\infty, \infty) \)
- **PDF:**
\[
f(x) = \frac{\Gamma\left(\frac{\nu + 1}{2}\right)}{\sqrt{\nu \pi} \; \Gamma\left(\frac{\nu}{2}\right)} \left(1 + \frac{x^2}{\nu}\right)^{-\frac{\nu + 1}{2}}
\]

---

### 8. F-Distribution \( F(d_1, d_2) \)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{1}{B\left(\frac{d_1}{2}, \frac{d_2}{2}\right)} \left(\frac{d_1}{d_2}\right)^{d_1/2} x^{\frac{d_1}{2} - 1} \left(1 + \frac{d_1}{d_2} x\right)^{-\frac{d_1 + d_2}{2}}
\]

---

### 9. Weibull Distribution \( \text{Weibull}(k, \lambda) \)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{k}{\lambda} \left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^k}
\]

---

### 10. Log-Normal Distribution \( \text{LogNormal}(\mu, \sigma) \)
- **Support:** \( x > 0 \)
- **PDF:**
\[
f(x) = \frac{1}{x \sigma \sqrt{2\pi}} \exp\left(-\frac{(\ln x - \mu)^2}{2 \sigma^2}\right)
\]

---

If you want pdfs for more specialized distributions or discrete distributions, just let me know!
User: definition of fisher information
Assistant: The **Fisher Information** measures the amount of information that an observable random variable \(X\) carries about an unknown parameter \(\theta\) upon which the probability of \(X\) depends. It quantifies the sensitivity of the likelihood function to changes in the parameter.

### Formal Definition

Suppose \(X\) is a random variable with probability density function (or probability mass function) \( f(x; \theta) \), where \(\theta\) is a parameter (scalar or vector). The Fisher information \( I(\theta) \) is defined as:

\[
I(\theta) = \mathbb{E}_\theta \left[ \left( \frac{\partial}{\partial \theta} \log f(X; \theta) \right)^2 \right]
\]

where the expectation is taken with respect to the distribution of \(X\) given \(\theta\).

Equivalently, under regularity conditions (allowing exchange of derivative and integral), Fisher information can be expressed as:

\[
I(\theta) = - \mathbb{E}_\theta \left[ \frac{\partial^2}{\partial \theta^2} \log f(X; \theta) \right]
\]

### Intuition

- \(\frac{\partial}{\partial \theta} \log f(X; \theta)\) is called the **score function**, representing the gradient of the log-likelihood.
- The Fisher information is the variance of the score, showing how much the log-likelihood changes with \(\theta\).
- Higher Fisher information means more precision in estimating \(\theta\) from data.

### Multivariate Case

If \(\theta\) is a vector parameter, the Fisher information becomes a matrix:

\[
[I(\theta)]_{ij} = \mathbb{E}_\theta \left[ \frac{\partial}{\partial \theta_i} \log f(X; \theta) \cdot \frac{\partial}{\partial \theta_j} \log f(X; \theta) \right]
\]

---

**Summary:**

\[
\boxed{
I(\theta) = \mathbb{E}_\theta \left[ \left(\frac{\partial}{\partial \theta} \log f(X; \theta) \right)^2 \right] = -\mathbb{E}_\theta \left[ \frac{\partial^2}{\partial \theta^2} \log f(X; \theta) \right]
}
\]
User: exp and variance of bernoulli
Assistant: For a Bernoulli random variable \( X \sim \text{Bern}(p) \), where \( p = P(X=1) \) and \( 1 - p = P(X=0) \):

### Expectation (Mean)
\[
\mathbb{E}[X] = p
\]

### Variance
\[
\mathrm{Var}(X) = p(1 - p)
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = p \\
\mathrm{Var}(X) = p(1 - p)
\end{cases}
}
\]
User: exp and variance of binomial
Assistant: For a Binomial random variable \( X \sim \text{Binomial}(n, p) \), where \( n \) is the number of independent trials and \( p \) is the success probability in each trial:

### Expectation (Mean)
\[
\mathbb{E}[X] = n p
\]

### Variance
\[
\mathrm{Var}(X) = n p (1 - p)
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = n p \\
\mathrm{Var}(X) = n p (1 - p)
\end{cases}
}
\]
User: exp and var of poisson
Assistant: For a Poisson random variable \( X \sim \text{Poisson}(\lambda) \), where \(\lambda > 0\) is the rate (or mean number of events):

### Expectation (Mean)
\[
\mathbb{E}[X] = \lambda
\]

### Variance
\[
\mathrm{Var}(X) = \lambda
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \lambda \\
\mathrm{Var}(X) = \lambda
\end{cases}
}
\]
User: exp and var geometric
Assistant: For a Geometric random variable \( X \), there are two common definitions depending on the parameterization:

---

### Definition 1: Number of **failures before** the first success

- Support: \( X = 0, 1, 2, \dots \)
- Parameter: Success probability \( p \in (0, 1) \)
- PMF: 
\[
P(X = k) = (1-p)^k p, \quad k = 0,1,2,\dots
\]

**Expectation and Variance:**
\[
\mathbb{E}[X] = \frac{1-p}{p}
\]
\[
\mathrm{Var}(X) = \frac{1-p}{p^2}
\]

---

### Definition 2: Number of **trials up to and including** the first success

- Support: \( X = 1, 2, 3, \dots \)
- Parameter: Success probability \( p \in (0, 1) \)
- PMF:
\[
P(X = k) = (1-p)^{k-1} p, \quad k = 1,2,3,\dots
\]

**Expectation and Variance:**
\[
\mathbb{E}[X] = \frac{1}{p}
\]
\[
\mathrm{Var}(X) = \frac{1-p}{p^2}
\]

---

**Summary:**

| Definition                          | Support           | \( \mathbb{E}[X] \)      | \( \mathrm{Var}(X) \)        |
|-----------------------------------|-------------------|--------------------------|------------------------------|
| Failures before first success      | \( 0,1,2,\dots \) | \( \frac{1-p}{p} \)      | \( \frac{1-p}{p^2} \)        |
| Trials until first success          | \( 1,2,3,\dots \) | \( \frac{1}{p} \)        | \( \frac{1-p}{p^2} \)        |

---

Let me know if you want details or derivations!
User: exp and var neg binom
Assistant: For a Negative Binomial random variable \( X \sim \text{NegBin}(r, p) \), there are two common parameterizations. Here, we consider the common one where:

- \(r > 0\) is the number of **successes** to be observed,
- \(p \in (0,1)\) is the probability of **success** on each trial,
- \(X\) counts the number of **failures** before the \(r\)-th success.

---

### Negative Binomial \(X =\) Number of failures before the \(r\)-th success

- Support: \(X = 0, 1, 2, \ldots\)

#### Expectation (Mean):
\[
\mathbb{E}[X] = \frac{r(1-p)}{p}
\]

#### Variance:
\[
\mathrm{Var}(X) = \frac{r(1-p)}{p^2}
\]

---

### Alternative parameterization: \(Y = X + r\) is the number of trials until the \(r\)-th success.

- Support: \(Y = r, r+1, r+2, \ldots\)

#### Expectation:
\[
\mathbb{E}[Y] = \frac{r}{p}
\]

#### Variance:
\[
\mathrm{Var}(Y) = \frac{r(1-p)}{p^2}
\]

---

### Notes:

- When \(r=1\), the Negative Binomial reduces to the Geometric distribution (failures before 1st success).
- The Negative Binomial describes the sum of \(r\) independent Geometric random variables (failures before success).
  
---

### Summary:

\[
\boxed{
\begin{cases}
X \sim \text{NegBin}(r,p): \text{failures before } r\text{-th success} \\
\mathbb{E}[X] = \dfrac{r(1-p)}{p} \\
\mathrm{Var}(X) = \dfrac{r(1-p)}{p^2}
\end{cases}
}
\]

If you want formulas or details for alternative parameterizations, let me know!
User: exp and var uniform continuous
Assistant: For a continuous random variable \( X \sim \text{Uniform}(a,b) \) with \( a < b \):

---

### Expectation (Mean)
\[
\mathbb{E}[X] = \frac{a+b}{2}
\]

---

### Variance
\[
\mathrm{Var}(X) = \frac{(b - a)^2}{12}
\]

---

### Summary:
\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \frac{a+b}{2} \\
\mathrm{Var}(X) = \frac{(b - a)^2}{12}
\end{cases}
}
\]
User: exp and var exponential
Assistant: For a continuous random variable \( X \sim \text{Exponential}(\lambda) \) with rate parameter \(\lambda > 0\), defined on \( x \geq 0 \):

---

### Expectation (Mean)
\[
\mathbb{E}[X] = \frac{1}{\lambda}
\]

---

### Variance
\[
\mathrm{Var}(X) = \frac{1}{\lambda^2}
\]

---

### Summary:
\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \frac{1}{\lambda} \\
\mathrm{Var}(X) = \frac{1}{\lambda^2}
\end{cases}
}
\]
User: now gamma
Assistant: For a Gamma random variable \( X \sim \Gamma(\alpha, \beta) \), where:
- \(\alpha > 0\) is the **shape** parameter,
- \(\beta > 0\) is the **rate** parameter (sometimes denoted as \(\theta = 1/\beta\) called scale),

and the PDF is:
\[
f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\beta x}, \quad x \geq 0,
\]

---

### Expectation (Mean)
\[
\mathbb{E}[X] = \frac{\alpha}{\beta}
\]

---

### Variance
\[
\mathrm{Var}(X) = \frac{\alpha}{\beta^2}
\]

---

### Notes:
- Sometimes the Gamma is parameterized using a **scale** parameter \(\theta = 1/\beta\) instead of rate:
  \[
  f(x) = \frac{1}{\Gamma(\alpha) \theta^\alpha} x^{\alpha - 1} e^{-x/\theta}
  \]
  In that case,
  \[
  \mathbb{E}[X] = \alpha \theta, \quad \mathrm{Var}(X) = \alpha \theta^2.
  \]

---

### Summary (rate parameterization):
\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \frac{\alpha}{\beta} \\
\mathrm{Var}(X) = \frac{\alpha}{\beta^2}
\end{cases}
}
\]

Let me know if you want details on other parameterizations or derivations!
User: now beta
Assistant: For a Beta random variable \( X \sim \text{Beta}(\alpha, \beta) \) with parameters \(\alpha > 0\) and \(\beta > 0\), defined on the interval \( [0, 1] \), the probability density function is:

\[
f(x) = \frac{x^{\alpha - 1} (1 - x)^{\beta - 1}}{B(\alpha, \beta)}, \quad 0 \leq x \leq 1,
\]
where \( B(\alpha, \beta) = \frac{\Gamma(\alpha) \Gamma(\beta)}{\Gamma(\alpha + \beta)} \) is the Beta function.

---

### Expectation (Mean)
\[
\mathbb{E}[X] = \frac{\alpha}{\alpha + \beta}
\]

---

### Variance
\[
\mathrm{Var}(X) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}
\]

---

### Summary:
\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \frac{\alpha}{\alpha + \beta} \\
\mathrm{Var}(X) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}
\end{cases}
}
\]

Let me know if you want more details or moments of other distributions!
User: guidelines for determining the rejection region of a hypothesis test given alpha
Assistant: When performing a hypothesis test at significance level \(\alpha\), determining the **rejection region** involves deciding which values of the test statistic will lead you to reject the null hypothesis \(H_0\).

Here are the general **guidelines** to determine the rejection region:

---

### 1. **Identify the Hypotheses and Test Type**

- **Null hypothesis:** \(H_0\)
- **Alternative hypothesis:** \(H_a\) (can be one-sided or two-sided)
  
Types of tests (assume test statistic \(T\)):

- **Right-tailed test:** \(H_a: \theta > \theta_0\)
- **Left-tailed test:** \(H_a: \theta < \theta_0\)
- **Two-tailed test:** \(H_a: \theta \neq \theta_0\)

---

### 2. **Specify the Significance Level**

\[
\alpha = P(\text{Type I error}) = P(\text{reject } H_0 \mid H_0 \text{ true})
\]

Common values: 0.05, 0.01, 0.10

---

### 3. **Find the Distribution of the Test Statistic under \(H_0\)**

Determine the sampling distribution of \(T\) assuming \(H_0\) is true. It could be:

- Normal,
- t-distribution,
- Chi-square,
- F-distribution,
- Others.

---

### 4. **Determine Critical Value(s) Corresponding to \(\alpha\)**

- **Right-tailed test:** Find \(c\) such that:

\[
P(T > c \mid H_0) = \alpha
\]

Rejection region:

\[
\{ T > c \}
\]

- **Left-tailed test:** Find \(c\) such that:

\[
P(T < c \mid H_0) = \alpha
\]

Rejection region:

\[
\{ T < c \}
\]

- **Two-tailed test:** Find critical values \(c_1, c_2\) such that:

\[
P(T < c_1 \mid H_0) = \frac{\alpha}{2}, \quad P(T > c_2 \mid H_0) = \frac{\alpha}{2}
\]

Rejection region:

\[
\{ T < c_1 \} \cup \{ T > c_2 \}
\]

---

### 5. **Express the Rejection Region**

Write explicitly the set of test statistic values where you reject \(H_0\).

---

### 6. **Use Tables or Statistical Software to Find Critical Values**

- For common distributions, critical values (quantiles) are tabulated.
- Alternatively, use software functions like `norm.ppf()`, `t.ppf()`, `chi2.ppf()`, etc.

---

### **Example Summary:**

| Test Type        | Rejection Region                      | Critical Value(s)                     |
|------------------|-------------------------------------|-------------------------------------|
| Right-tailed     | \( T > c \)                        | \( c = F_T^{-1}(1-\alpha) \)        |
| Left-tailed      | \( T < c \)                        | \( c = F_T^{-1}(\alpha) \)           |
| Two-tailed      | \( T < c_1 \) or \( T > c_2 \)    | \( c_1 = F_T^{-1}(\alpha/2), \quad c_2 = F_T^{-1}(1 - \alpha/2) \) |

where \(F_T^{-1}\) denotes the quantile (inverse CDF) function of the test statistic \(T\) under the null hypothesis.

---

### **Additional Tips**

- Be clear about the direction of the alternative hypothesis.
- Check assumptions to ensure the test statistic distribution is valid.
- For discrete test statistics, rejection regions may not exactly match \(\alpha\) but should be as close as possible without exceeding \(\alpha\).
- For complex tests, consider using p-values rather than fixed rejection regions.

---

If you'd like, I can provide examples for specific hypothesis tests!
User: definition of bias and consistency
Assistant: Here are the definitions of **bias** and **consistency** in the context of statistical estimation:

---

### 1. Bias

**Bias** of an estimator \(\hat{\theta}\) of a parameter \(\theta\) measures the difference between the estimator's expected value and the true parameter value.

\[
\boxed{
\text{Bias}(\hat{\theta}) = \mathbb{E}[\hat{\theta}] - \theta
}
\]

- **Unbiased estimator:** \(\mathbb{E}[\hat{\theta}] = \theta\), so \(\text{Bias}(\hat{\theta}) = 0\).
- **Biased estimator:** If \(\text{Bias}(\hat{\theta}) \neq 0\).

**Interpretation:**  
Bias indicates systematic error in estimation. An unbiased estimator, on average, hits the true parameter value; a biased estimator systematically underestimates or overestimates.

---

### 2. Consistency

An estimator \(\hat{\theta}_n\) (based on a sample of size \(n\)) is **consistent** for \(\theta\) if it converges in probability to the true parameter as the sample size grows:

\[
\boxed{
\hat{\theta}_n \xrightarrow{p} \theta \quad \text{as } n \to \infty
}
\]
which means that for any \(\varepsilon > 0\):

\[
\lim_{n \to \infty} P(|\hat{\theta}_n - \theta| > \varepsilon) = 0
\]

- Roughly, the estimator becomes arbitrarily close to \(\theta\) with high probability as more data is collected.

---

### **Summary:**

| Property   | Definition                                      | Interpretation                        |
|------------|------------------------------------------------|-------------------------------------|
| **Bias**  | \(\mathbb{E}[\hat{\theta}] - \theta\)          | Systematic deviation from true value |
| **Consistency** | \(\hat{\theta}_n \xrightarrow{p} \theta\) as \(n \to \infty\) | Estimator converges to true value with large samples |

---

If you want, I can also explain related concepts like efficiency or mean squared error!
User: uniformly most powerful test criteria
Assistant: *Uniformly Most Powerful (UMP) Test:**

A **Uniformly Most Powerful (UMP)** test is a hypothesis test that maximizes the power (probability of correctly rejecting the null hypothesis when the alternative is true) **for all parameter values** in the alternative hypothesis class, subject to a fixed size \(\alpha\) (type I error rate).

---

### Criteria for UMP Tests:

1. **Definition:**

For testing

\[
H_0: \theta = \theta_0 \quad \text{vs} \quad H_1: \theta \in \Theta_1,
\]

a test \(\phi^*\) is **UMP at level \(\alpha\)** if:

- \(\mathbb{E}_{\theta_0}[\phi^*(X)] = \alpha\) (test has size \(\alpha\)),
- For **every** other test \(\phi\) of size \(\leq \alpha\), and for **every** \(\theta \in \Theta_1\),
\[
\mathbb{E}_{\theta}[\phi^*(X)] \geq \mathbb{E}_{\theta}[\phi(X)].
\]

In words, \(\phi^*\) has the greatest power uniformly over all \(\theta \in \Theta_1\).

---

2. **Neyman-Pearson Lemma (Basis for UMP for Simple vs Simple Tests):**

- For testing \(H_0: \theta = \theta_0\) vs \(H_1: \theta = \theta_1\),
- The most powerful test rejects \(H_0\) for large values of the likelihood ratio:
\[
\Lambda(x) = \frac{f(x; \theta_1)}{f(x; \theta_0)} > k,
\]
where \(k\) is chosen to satisfy size \(\alpha\).

---

3. **Extension to Composite Alternatives (Conditions for UMP Existence):**

- UMP tests often exist if the family of distributions has a **Monotone Likelihood Ratio (MLR)** property in a statistic \(T(X)\).
- The test statistic \(T(X)\) satisfies that for \(\theta_1 > \theta_0\),
\[
\frac{f_{T}(t; \theta_1)}{f_{T}(t; \theta_0)} \text{ is non-decreasing in } t,
\]
where \(f_T\) is the density of \(T\).

- Under MLR, a test that rejects \(H_0\) if \(T(X) > c\) for a certain cutoff \(c\) is UMP for testing:

\[
H_0: \theta \leq \theta_0 \quad \text{vs} \quad H_1: \theta > \theta_0.
\]

---

4. **UMP Tests May Not Exist:**

- In many cases (especially two-sided tests or multidimensional parameters), no UMP test exists.
- One resorts to:
  - UMP unbiased tests,
  - Likelihood ratio tests,
  - Other optimality criteria.

---

### **Summary**

| Criterion                 | Details                                                                                      |
|---------------------------|----------------------------------------------------------------------------------------------|
| **UMP test**              | Maximizes power uniformly over all parameter values in \(H_1\) at fixed significance level \(\alpha\). |
| **Exists when**           | A monotone likelihood ratio (MLR) condition holds in a scalar test statistic \(T\).          |
| **Test form**             | Reject \(H_0\) if \(T(X)\) exceeds a critical value \(c\).                                  |
| **Neyman-Pearson Lemma**  | Provides Most Powerful test for simple hypotheses; basis for UMP under MLR condition.        |
| **Often does not exist**  | For two-sided or multidimensional composite alternatives.                                   |

---

If you want, I can provide examples or explain MLR
User: posterior predictive distribution
Assistant: