User: list of common continuous prob distributions and their density functions
Assistant: Here is a list of common continuous probability distributions along with their probability density functions (pdf):

### 1. Uniform Distribution \( \text{Uniform}(a,b) \)
- **Support:** \( x \in [a,b] \)
- **PDF:** 
\[
f(x) = \frac{1}{b - a}, \quad a \leq x \leq b
\]

---

### 2. Normal (Gaussian) Distribution \( \mathcal{N}(\mu, \sigma^2) \)
- **Support:** \( x \in (-\infty, \infty) \)
- **PDF:** 
\[
f(x) = \frac{1}{\sqrt{2\pi \sigma^2}} \exp\left(-\frac{(x - \mu)^2}{2\sigma^2}\right)
\]

---

### 3. Exponential Distribution \( \text{Exp}(\lambda) \)
- **Support:** \( x \geq 0 \)
- **PDF:** 
\[
f(x) = \lambda e^{-\lambda x}
\]

---

### 4. Gamma Distribution \( \Gamma(\alpha, \beta) \) (shape \(\alpha > 0\), rate \(\beta > 0\))
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{\beta^\alpha}{\Gamma(\alpha)} x^{\alpha - 1} e^{-\beta x}
\]

---

### 5. Beta Distribution \( \text{Beta}(\alpha, \beta) \)
- **Support:** \( x \in [0,1] \)
- **PDF:** 
\[
f(x) = \frac{x^{\alpha - 1} (1 - x)^{\beta - 1}}{B(\alpha, \beta)}
\]
where \( B(\alpha, \beta) = \frac{\Gamma(\alpha) \Gamma(\beta)}{\Gamma(\alpha + \beta)} \)

---

### 6. Chi-Square Distribution (\(\chi^2_k\)) — Special case of Gamma with \(\alpha = k/2, \beta = 1/2\)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{1}{2^{k/2} \Gamma(k/2)} x^{\frac{k}{2} - 1} e^{-x/2}
\]

---

### 7. Student's t-Distribution \( t_\nu \)
- **Support:** \( x \in (-\infty, \infty) \)
- **PDF:**
\[
f(x) = \frac{\Gamma\left(\frac{\nu + 1}{2}\right)}{\sqrt{\nu \pi} \; \Gamma\left(\frac{\nu}{2}\right)} \left(1 + \frac{x^2}{\nu}\right)^{-\frac{\nu + 1}{2}}
\]

---

### 8. F-Distribution \( F(d_1, d_2) \)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{1}{B\left(\frac{d_1}{2}, \frac{d_2}{2}\right)} \left(\frac{d_1}{d_2}\right)^{d_1/2} x^{\frac{d_1}{2} - 1} \left(1 + \frac{d_1}{d_2} x\right)^{-\frac{d_1 + d_2}{2}}
\]

---

### 9. Weibull Distribution \( \text{Weibull}(k, \lambda) \)
- **Support:** \( x \geq 0 \)
- **PDF:**
\[
f(x) = \frac{k}{\lambda} \left(\frac{x}{\lambda}\right)^{k-1} e^{-(x/\lambda)^k}
\]

---

### 10. Log-Normal Distribution \( \text{LogNormal}(\mu, \sigma) \)
- **Support:** \( x > 0 \)
- **PDF:**
\[
f(x) = \frac{1}{x \sigma \sqrt{2\pi}} \exp\left(-\frac{(\ln x - \mu)^2}{2 \sigma^2}\right)
\]

---

If you want pdfs for more specialized distributions or discrete distributions, just let me know!
User: definition of fisher information
Assistant: The **Fisher Information** measures the amount of information that an observable random variable \(X\) carries about an unknown parameter \(\theta\) upon which the probability of \(X\) depends. It quantifies the sensitivity of the likelihood function to changes in the parameter.

### Formal Definition

Suppose \(X\) is a random variable with probability density function (or probability mass function) \( f(x; \theta) \), where \(\theta\) is a parameter (scalar or vector). The Fisher information \( I(\theta) \) is defined as:

\[
I(\theta) = \mathbb{E}_\theta \left[ \left( \frac{\partial}{\partial \theta} \log f(X; \theta) \right)^2 \right]
\]

where the expectation is taken with respect to the distribution of \(X\) given \(\theta\).

Equivalently, under regularity conditions (allowing exchange of derivative and integral), Fisher information can be expressed as:

\[
I(\theta) = - \mathbb{E}_\theta \left[ \frac{\partial^2}{\partial \theta^2} \log f(X; \theta) \right]
\]

### Intuition

- \(\frac{\partial}{\partial \theta} \log f(X; \theta)\) is called the **score function**, representing the gradient of the log-likelihood.
- The Fisher information is the variance of the score, showing how much the log-likelihood changes with \(\theta\).
- Higher Fisher information means more precision in estimating \(\theta\) from data.

### Multivariate Case

If \(\theta\) is a vector parameter, the Fisher information becomes a matrix:

\[
[I(\theta)]_{ij} = \mathbb{E}_\theta \left[ \frac{\partial}{\partial \theta_i} \log f(X; \theta) \cdot \frac{\partial}{\partial \theta_j} \log f(X; \theta) \right]
\]

---

**Summary:**

\[
\boxed{
I(\theta) = \mathbb{E}_\theta \left[ \left(\frac{\partial}{\partial \theta} \log f(X; \theta) \right)^2 \right] = -\mathbb{E}_\theta \left[ \frac{\partial^2}{\partial \theta^2} \log f(X; \theta) \right]
}
\]
User: exp and variance of bernoulli
Assistant: For a Bernoulli random variable \( X \sim \text{Bern}(p) \), where \( p = P(X=1) \) and \( 1 - p = P(X=0) \):

### Expectation (Mean)
\[
\mathbb{E}[X] = p
\]

### Variance
\[
\mathrm{Var}(X) = p(1 - p)
\]

---

**Summary:**

\[
\boxed{
\begin{cases}
\mathbb{E}[X] = p \\
\mathrm{Var}(X) = p(1 - p)
\end{cases}
}
\]
User: exp and variance of binomial
Assistant: