Problem 1 (qiuzhen-qualify 2024-fall)

(10 points) Let $U_1, U_2, \dots$ be independent identically distributed (i.i.d.) random variables uniformly distributed on $[0,1]$, and define $$S_n = \sum_{k=1}^n U_k.$$
(a) Calculate $\mathbb{E}[(S_n)^4]$.
(b) Determine the distribution of $S_3$.

Problem 2 (qiuzhen-qualify 2024-fall)

(10 points) Let $(Y_n)_{n\ge 1}$ be a sequence of real-valued random variables such that $$\sqrt{n}\,(Y_n-a) \xrightarrow{(d)}_{n\to\infty} \mathcal{N}(0,\sigma^2),$$ where $\mathcal{N}(0,\sigma^2)$ stands for the normal distribution with $\sigma\ne 0$.
(a) Let $g:\mathbb{R}\to\mathbb{R}$ be a function such that it is differentiable at $a$ and $g'(a)\ne 0$. Prove that $$\sqrt{n}\,(g(Y_n)-g(a)) \xrightarrow{(d)}_{n\to\infty} \mathcal{N}\big(0,(g'(a))^2\sigma^2\big).$$
(b) Fix $p\in(0,1)$. For $n\in\mathbb{N}$, let $Z_n \stackrel{(d)}{=} \mathrm{Bin}(n,p)$ be a binomial random variable. Prove that $$\sqrt{n}\Big(\ln\Big(\frac{Z_n}{n}\Big)-\ln p\Big) \xrightarrow{(d)}_{n\to\infty} \mathcal{N}\!\left(0,\frac{1-p}{p}\right).$$

Problem 3 (qiuzhen-qualify 2024-fall)

(10 points) Let $X$ be a real-valued random variable. Prove that the following properties are equivalent in the sense that the parameters $K_i>0$ appearing in these properties differ from each other by at most an absolute constant factor.
(a) The tails of $X$ satisfy that $$\mathbb{P}(|X|\ge t)\le 2\exp(-t/K_1)\quad \text{for all } t\ge 0.$$
(b) The moments of $X$ satisfy that $$\mathbb{E}(|X|^p)\le (K_2 p)^p\quad \text{for all } p\in\mathbb{N}.$$
(c) The moment generating function of $|X|$ is bounded at some point, i.e., $$\mathbb{E}\exp\big(|X|/K_3\big)\le 2$$ for some $K_3>0$.
Hint: You can use Stirling’s approximations: $$e(n/e)^n\le n!\le e\,n(n/e)^n\quad \text{for all } n\in\mathbb{N}$$ and $$n! = (1+o(1))\sqrt{2\pi n}\,(n/e)^n\quad \text{for large } n.$$

Problem 4 (qiuzhen-qualify 2024-fall)

(10 points) Let $\mathsf{T}_d$ be an infinite $d$-regular tree, where every node has degree $d$. On the other hand, let $\mathsf{T}_b$ be an infinite $b$-ary tree with root $o$. In other words, $\mathsf{T}_b$ is an infinite tree where every node has $b$ children nodes and every non-root node has one parent node. Below is an illustration of a binary tree with $b=2$.

Consider the simple random walk $X_n$ on $\mathsf{T}_d$ and simple random walk $Y_n$ on $\mathsf{T}_b$, where at each step, the walker moves to the neighbor nodes with equal probability.
(a) For each $d\in\mathbb{N}$ with $d\ge 2$, determine whether the simple random walk $X_n$ on $\mathsf{T}_d$ is recurrent or transient. Prove your claim.
(b) For each $b\in\mathbb{N}$, determine whether the simple random walk $Y_n$ on $\mathsf{T}_b$ is recurrent or transient. Prove your claim. (Hint: Use part (a).)

Problem 5 (qiuzhen-qualify 2024-fall)

(15 points) Let $X_1,X_2,\dots$ be i.i.d. random variables with exponential distribution: $\mathbb{P}(X_k>x)=e^{-x}$ for $x\ge 0$. Define $$M_n := \sum_{k=1}^n \frac{X_k}{k}.$$
(a) Prove that $(M_n-\ln n)_{n\in\mathbb{N}}$ converges to a limit $Y$ almost surely.
(b) Prove that, for every $p\in(0,1)$, $$\left(\frac{\exp(p\,M_n)}{n^p}\right)_{n\in\mathbb{N}}$$ converges to a limit $Z$ in $L^1$.

Problem 6 (qiuzhen-qualify 2024-fall)

(15 points) Let $B_t$ be a one-dimensional (1D) standard Brownian motion started from $0$.
(a) Consider a Brownian motion $X_t = x + B_t$ started at some $x>0$. For any $t>0$ and $b>a>0$, compute the probability of $X_t\in[a,b]$ conditioning on that $X_t$ does not hit zero between $0$ and $t$, i.e., $$\mathbb{P}\Big( X_t\in[a,b] \;\Big|\; \min_{0\le s\le t} X_s > 0 \Big).$$

Problem 6(b) (qiuzhen-qualify 2024-fall)

A Brownian bridge $W_t$ on $[0,1]$ is a 1D standard Brownian motion $B_t$ subject to the condition that $B_1=0$. In other words, $W_t=(B_t\mid B_1=0)$ is a continuous-time Gaussian process whose probability distribution is the conditional probability distribution of $B_t$ conditioning on $B_1=0$. A 1D Gaussian free field (GFF) $h_t$ on $[0,1]$ with zero boundary is a continuous-time Gaussian process subject to the zero boundary condition $h_0=h_1=0$ and has zero mean $\mathbb{E} h_t=0$, $t\in[0,1]$, and covariances

$$\mathbb{E}(h_t h_s)=G(t,s),\quad t,s\in[0,1].$$

Here, $G(t,s)$ is the Green’s function of the Laplace operator $-\Delta$, i.e., $G(t,s)$ is the unique continuous function such that for any smooth test function $f\in C_c^{\infty}(0,1)$,

$$\int_0^1 G(t,s)\,\frac{\partial^2}{\partial t^2} f(t)\,dt=-f(s)\quad\text{and}\quad G(0,s)=G(1,s)=0.$$

Prove that the process $(W_t:t\in[0,1])$ has the same distribution as the process $(h_t:t\in[0,1])$ in the sense that for any fixed $0\le t_1<t_2<\dots<t_n\le 1$ and Borel sets $O_1,O_2,\dots,O_n$,

$$\mathbb{P}(W_{t_1}\in O_1,\dots,W_{t_n}\in O_n)=\mathbb{P}(h_{t_1}\in O_1,\dots,h_{t_n}\in O_n).$$

(Hint: Find the explicit form of the function $G(t,s)$ and calculate $\mathbb{E}(W_t W_s)$.)

Problem 7 (qiuzhen-qualify 2024-fall)

Let $X_1,\ldots,X_n$ be an iid sample from $N(\mu,1)$ with $\mu$ unknown. Unfortunately, one forgets to record $X_1,\ldots,X_n$ in a study and only records $Y=(Y_1,\ldots,Y_n)$ where $Y_i=I(X_i<0)$ and $I(\cdot)$ is the indicator function.

(a) Derive the MLE of $\mu$ based on the observed data $\mathbf{Y}$.

(b) Construct a size $\alpha$ uniformly most powerful (UMP) test for testing $H_0: \mu\le \mu_0$ versus $H_1: \mu>\mu_0$ based on the observed data $\mathbf{Y}$.

(c) Describe how to construct a $(1-\alpha)$ confidence interval for $\mu$ based on the observed data $\mathbf{Y}$.

Problem 8 (qiuzhen-qualify 2024-fall)

Consider the following linear model $Y_i=z_i^T\beta+\epsilon_i$, $i=1,\ldots,n$. $z_1,\ldots,z_n\in\mathbb{R}^d$ are fixed and given, and $\beta\in\mathbb{R}^d$ is unknown. $\epsilon_i$’s are random variables satisfying the Gauss–Markov assumptions that $\mathbb{E}[\epsilon_i]=0$, $\operatorname{Var}[\epsilon_i]=\sigma^2$ and $\operatorname{Cov}(\epsilon_i,\epsilon_j)=0$, $\forall i\ne j$. Let $\mathbf{Y}=(Y_1,\ldots,Y_n)^T$, and

$$\mathbf{Z}=\begin{pmatrix} z_1^T\\ z_2^T\\ \vdots\\ z_n^T \end{pmatrix}$$

be the $n$ by $d$ design matrix.

(a) Let $\hat\beta$ be the least squares estimate of $\beta$ which is given by $\hat\beta=(\mathbf{Z}^T\mathbf{Z})^{-1}\mathbf{Z}^T\mathbf{Y}$. Let $\theta=b^T\beta$ where $b\in\mathbb{R}^d$ is a known vector. Write down the mean and the variance of $\hat\theta$ where $\hat\theta=b^T\hat\beta$. Further, prove that under the Gauss–Markov assumptions, the estimator $\hat\theta$ has the smallest variance among all linear unbiased estimator of $\theta$. Here linear unbiased estimator we mean estimator in the form of $c^T\mathbf{Y}$ and is unbiased for $\theta$.

(b) Further assume that $(\epsilon_1,\ldots,\epsilon_n)$ are iid from $N(0,\sigma^2)$ with $\sigma^2$ known. Derive the information matrix $I(\beta)$.

Problem 9 (qiuzhen-qualify 2024-fall)

Suppose $X_1,\ldots,X_n$ are IID from the uniform distribution on $[0,\theta]$ for some unknown $\theta>0$. Fix $t\in(0,\theta)$. Consider two estimators of $\mathbb{P}(X_1\le t)$: 

$$F_n(t)=\frac{1}{n}\sum_{i=1}^n I(X_i\le t)\quad\text{and}\quad T_n(t)=\frac{t}{2\bar X},$$

where $\bar X$ is the sample mean.

(a) Find the asymptotic distributions of the two estimators.

(b) For what value of $t$ will the first estimator have a smaller asymptotic variance than the second estimator?

(c) Let $\theta=1$. For the $F_n(t)$ defined above, find the asymptotic distribution of $nF_n(n^{-1/2})-\sqrt{n}.$

Problem 10 (qiuzhen-qualify 2024-fall)

Let $X_1,\ldots,X_n$ ($n\ge 2$) be iid from $N(\mu,\sigma^2)$ distribution with $\mu\ge 0$ and $\sigma>0$ being the unknown parameters. Let $\bar X$ and $S^2$ be the sample mean and sample variance, respectively. Recall $\chi_k^2$ has probability density function

$$\frac{1}{2^{k/2}\,\Gamma(k/2)}\,x^{k/2-1}e^{-x/2},\quad x\ge 0.$$

(a) Show $\bar X$ and $S^2$ are independent.

(b) Find UMVUE of $\mu/\sigma$ if it exists.

(c) Is $\bar X$ admissible for estimating $\mu$ under the square error loss? Prove your assertion.

Problem 11 (qiuzhen-qualify 2024-fall)

Let $X_1,\ldots,X_n$ be an iid sample from $\operatorname{Uniform}[\theta,\,\theta+|\theta|]$ where $\theta\ne 0$.

(a) Derive the method of moments estimator of $\theta$.

(b) Derive the MLE of $\theta$, $\hat\theta$.

(c) Is $\hat\theta$ a consistent estimator of $\theta$? Please explain your answer.