User: list of common continuous prob distributions and their density functions
Assistant: Here is a list of common continuous probability distributions along with their probability density functions (pdf):

### 1. Uniform Distribution \( \text{Uniform}(a,b) \)
- **Support:** \( x \in [a,b] \)
- **PDF:** 
\[
f(x) = \frac{1}{b - a}, \quad a \leq x \leq b
\]

---

### 2. Normal (Gaussian) Distribution \( \mathcal{N}(\mu, \sigma^2) \)
- **Support:** \( x \in (-\infty, \infty) \)
- **PDF:** 
\[
f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right)
\]

---

### 3. Exponential Distribution \( \text{Exp}(\lambda) \)
- **Support:** \( x \geq 0 \)
- **PDF:** 
\[
f(x) = \lambda e^{-\lambda x}
\]

---

### 4. Gamma Distribution \( \Gamma(\alpha, \beta) \) (shape \(\alpha > 0\), rate \(\beta > 0\))
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\beta x}
\]

---

### 5. Beta Distribution \( \text{Beta}(\alpha, \beta) \)
- **Support:** \( x \in [0,1] \)
- **PDF:** 
\[
f(x) = \frac{x^{\alpha - 1} (1 - x)^{\beta - 1}}{B(\alpha, \beta)}
\]
where \( B(\alpha, \beta) = \frac{\Gamma(\alpha) \Gamma(\beta)}{\Gamma(\alpha + \beta)} \)

---

### 6. Chi-Square Distribution (\(\chi^2_k\)) — Special case of Gamma with \(\alpha = k/2, \beta = 1/2\)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{1}{2^{k/2} \Gamma(k/2)} x^{\frac{k}{2} - 1} e^{-x/2}
\]

---

### 7. Student's t-Distribution \( t_\nu \)
- **Support:** \( x \in (-\infty, \infty) \)
- **PDF:**
\[
f(x) = \frac{\Gamma\left(\frac{\nu + 1}{2}\right)}{\sqrt{\nu \pi} \; \Gamma\left(\frac{\nu}{2}\right)} \left(1 + \frac{x^2}{\nu}\right)^{-\frac{\nu + 1}{2}}
\]

---

### 8. F-Distribution \( F(d_1, d_2) \)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{1}{B\left(\frac{d_1}{2}, \frac{d_2}{2}\right)} \left(\frac{d_1}{d_2}\right)^{d_1/2} x^{\frac{d_1}{2} - 1} \left(1 + \frac{d_1}{d_2} x\right)^{-\frac{d_1 + d_2}{2}}
\]

---

### 9. Weibull Distribution \( \text{Weibull}(k, \lambda) \)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{k}{\lambda} \left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^k}
\]

---

### 10. Log-Normal Distribution \( \text{LogNormal}(\mu, \sigma) \)
- **Support:** \( x > 0 \)
- **PDF:**
\[
f(x) = \frac{1}{x \sigma \sqrt{2\pi}} \exp\left(-\frac{(\ln x - \mu)^2}{2 \sigma^2}\right)
\]

---

If you want pdfs for more specialized distributions or discrete distributions, just let me know!
User: definition of fisher information
Assistant: The **Fisher Information** measures the amount of information that an observable random variable \(X\) carries about an unknown parameter \(\theta\) upon which the probability of \(X\) depends. It quantifies the sensitivity of the likelihood function to changes in the parameter.

### Formal Definition

Suppose \(X\) is a random variable with probability density function (or probability mass function) \( f(x; \theta) \), where \(\theta\) is a parameter (scalar or vector). The Fisher information \( I(\theta) \) is defined as:

\[
I(\theta) = \mathbb{E}_\theta \left[ \left( \frac{\partial}{\partial \theta} \log f(X; \theta) \right)^2 \right]
\]

where the expectation is taken with respect to the distribution of \(X\) given \(\theta\).

Equivalently, under regularity conditions (allowing exchange of derivative and integral), Fisher information can be expressed as:

\[
I(\theta) = - \mathbb{E}_\theta \left[ \frac{\partial^2}{\partial \theta^2} \log f(X; \theta) \right]
\]

### Intuition

- \(\frac{\partial}{\partial \theta} \log f(X; \theta)\) is called the **score function**, representing the gradient of the log-likelihood.
- The Fisher information is the variance of the score, showing how much the log-likelihood changes with \(\theta\).
- Higher Fisher information means more precision in estimating \(\theta\) from data.

### Multivariate Case

If \(\theta\) is a vector parameter, the Fisher information becomes a matrix:

\[
[I(\theta)]_{ij} = \mathbb{E}_\theta \left[ \frac{\partial}{\partial \theta_i} \log f(X; \theta) \cdot \frac{\partial}{\partial \theta_j} \log f(X; \theta) \right]
\]

---

**Summary:**

\[
\boxed{
I(\theta) = \mathbb{E}_\theta \left[ \left(\frac{\partial}{\partial \theta} \log f(X; \theta) \right)^2 \right] = -\mathbb{E}_\theta \left[ \frac{\partial^2}{\partial \theta^2} \log f(X; \theta) \right]
}
\]
User: exp and variance of bernoulli
Assistant: For a Bernoulli random variable \( X \sim \text{Bern}(p) \), where \( p = P(X=1) \) and \( 1 - p = P(X=0) \):

### Expectation (Mean)
\[
\mathbb{E}[X] = p
\]

### Variance
\[
\mathrm{Var}(X) = p(1 - p)
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = p \\
\mathrm{Var}(X) = p(1 - p)
\end{cases}
}
\]
User: exp and variance of binomial
Assistant: For a Binomial random variable \( X \sim \text{Binomial}(n, p) \), where \( n \) is the number of independent trials and \( p \) is the success probability in each trial:

### Expectation (Mean)
\[
\mathbb{E}[X] = n p
\]

### Variance
\[
\mathrm{Var}(X) = n p (1 - p)
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = n p \\
\mathrm{Var}(X) = n p (1 - p)
\end{cases}
}
\]
User: exp and var of poisson
Assistant: For a Poisson random variable \( X \sim \text{Poisson}(\lambda) \), where \(\lambda > 0\) is the rate (or mean number of events):

### Expectation (Mean)
\[
\mathbb{E}[X] = \lambda
\]

### Variance
\[
\mathrm{Var}(X) = \lambda
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \lambda \\
\mathrm{Var}(X) = \lambda
\end{cases}
}
\]
User: exp and var geometric
Assistant: For a Geometric random variable \( X \), there are two common definitions depending on the parameterization:

---

### Definition 1: Number of **failures before** the first success

- Support: \( X = 0, 1, 2, \dots \)
- Parameter: Success probability \( p \in (0, 1) \)
- PMF: 
\[
P(X = k) = (1-p)^k p, \quad k = 0,1,2,\dots
\]

**Expectation and Variance:**
\[
\mathbb{E}[X] = \frac{1-p}{p}
\]
\[
\mathrm{Var}(X) = \frac{1-p}{p^2}
\]

---

### Definition 2: Number of **trials up to and including** the first success

- Support: \( X = 1, 2, 3, \dots \)
- Parameter: Success probability \( p \in (0, 1) \)
- PMF:
\[
P(X = k) = (1-p)^{k-1} p, \quad k = 1,2,3,\dots
\]

**Expectation and Variance:**
\[
\mathbb{E}[X] = \frac{1}{p}
\]
\[
\mathrm{Var}(X) = \frac{1-p}{p^2}
\]

---

**Summary:**

| Definition                          | Support           | \( \mathbb{E}[X] \)      | \( \mathrm{Var}(X) \)        |
|-----------------------------------|-------------------|--------------------------|------------------------------|
| Failures before first success      | \( 0,1,2,\dots \) | \( \frac{1-p}{p} \)      | \( \frac{1-p}{p^2} \)        |
| Trials until first success          | \( 1,2,3,\dots \) | \( \frac{1}{p} \)        | \( \frac{1-p}{p^2} \)        |

---

Let me know if you want details or derivations!
User: exp and var neg binom
Assistant: For a Negative Binomial random variable \( X \sim \text{NegBin}(r, p) \), there are two common parameterizations. Here, we consider the common one where:

- \(r > 0\) is the number of **successes** to be observed,
- \(p \in (0,1)\) is the probability of **success** on each trial,
- \(X\) counts the number of **failures** before the \(r\)-th success.

---

### Negative Binomial \(X =\) Number of failures before the \(r\)-th success

- Support: \(X = 0, 1, 2, \ldots\)

#### Expectation (Mean):
\[
\mathbb{E}[X] = \frac{r(1-p)}{p}
\]

#### Variance:
\[
\mathrm{Var}(X) = \frac{r(1-p)}{p^2}
\]

---

### Alternative parameterization: \(Y = X + r\) is the number of trials until the \(r\)-th success.

- Support: \(Y = r, r+1, r+2, \ldots\)

#### Expectation:
\[
\mathbb{E}[Y] = \frac{r}{p}
\]

#### Variance:
\[
\mathrm{Var}(Y) = \frac{r(1-p)}{p^2}
\]

---

### Notes:

- When \(r=1\), the Negative Binomial reduces to the Geometric distribution (failures before 1st success).
- The Negative Binomial describes the sum of \(r\) independent Geometric random variables (failures before success).
  
---

### Summary:

\[
\boxed{
\begin{cases}
X \sim \text{NegBin}(r,p): \text{failures before } r\text{-th success} \\
\mathbb{E}[X] = \dfrac{r(1-p)}{p} \\
\mathrm{Var}(X) = \dfrac{r(1-p)}{p^2}
\end{cases}
}
\]

If you want formulas or details for alternative parameterizations, let me know!
User: exp and var uniform continuous
Assistant: For a continuous random variable \( X \sim \text{Uniform}(a,b) \) with \( a < b \):

---

### Expectation (Mean)
\[
\mathbb{E}[X] = \frac{a+b}{2}
\]

---

### Variance
\[
\mathrm{Var}(X) = \frac{(b - a)^2}{12}
\]

---

### Summary:
\[
\boxed{
\begin{cases}
\mathbb{E}[X] = \frac{a+b}{2} \\
\mathrm{Var}(X) = \frac{(b - a)^2}{12}
\end{cases}
}
\]
User: exp and var exponential
Assistant: