Miracle-dz/UniFAR-FTUs huggingface dataset:
Parameter expansion in local-shrinkage models

This paper considers the problem of using MCMC to fit sparse Bayesian models based on normal scale-mixture priors. Examples of this framework include the Bayesian LASSO and the horseshoe prior. We study the usefulness of parameter expansion (PX) for improving convergence in such models, which is notoriously slow when the global variance component is near zero. Our conclusion is that parameter expansion does improve matters in LASSO-type models, but only modestly. In most cases this improvement, while noticeable, is less than what might be expected, especially compared to the improvements that PX makes possible for models very similar to those considered here. We give some examples, and we attempt to provide some intuition as to why this is so. We also describe how slice sampling may be used to update the global variance component. In practice, this approach seems to perform almost as well as parameter expansion. As a practical matter, however, it is perhaps best viewed not as a replacement for PX, but as a tool for expanding the class of models to which PX is applicable.
Multivariate integer-valued autoregressive models applied to earthquake counts

In various situations in the insurance industry, in finance, in epidemiology, etc., one needs to represent the joint evolution of the number of occurrences of an event. In this paper, we present a multivariate integer-valued autoregressive (MINAR) model, derive its properties and apply the model to earthquake occurrences across various pairs of tectonic plates. The model is an extension of Karlis & Pedelis (2011) where cross autocorrelation (spatial contagion in a seismic context) is considered. We fit various bivariate count models and find that for many contiguous tectonic plates, spatial contagion is significant in both directions. Furthermore, ignoring cross autocorrelation can underestimate the potential for high numbers of occurrences over the short-term. Our overall findings seem to further confirm Parsons & Velasco (2001).
A divergence formula for regularization methods with an L2 constraint

We derive a divergence formula for a group of regularization methods with an L2 constraint. The formula is useful for regularization parameter selection, because it provides an unbiased estimate for the number of degrees of freedom. We begin with deriving the formula for smoothing splines and then extend it to other settings such as penalized splines, ridge regression, and functional linear regression.
Static Parameter Estimation for ABC Approximations of Hidden Markov Models

In this article we focus on Maximum Likelihood estimation (MLE) for the static parameters of hidden Markov models (HMMs). We will consider the case where one cannot or does not want to compute the conditional likelihood density of the observation given the hidden state because of increased computational complexity or analytical intractability. Instead we will assume that one may obtain samples from this conditional likelihood and hence use approximate Bayesian computation (ABC) approximations of the original HMM. ABC approximations are biased, but the bias can be controlled to arbitrary precision via a parameter \epsilon>0; the bias typically goes to zero as \epsilon \searrow 0. We first establish that the bias in the log-likelihood and gradient of the log-likelihood of the ABC approximation, for a fixed batch of data, is no worse than \mathcal{O}(n\epsilon), n being the number of data; hence, for computational reasons, one might expect reasonable parameter estimates using such an ABC approximation. Turning to the computational problem of estimating $\theta$, we propose, using the ABC-sequential Monte Carlo (SMC) algorithm in Jasra et al. (2012), an approach based upon simultaneous perturbation stochastic approximation (SPSA). Our method is investigated on two numerical examples
Analysis of spatial distribution of marker expression in cells using boundary distance plots

Boundary distance (BD) plotting is a technique for making orientation invariant comparisons of the spatial distribution of biochemical markers within and across cells/nuclei. Marker expression is aggregated over points with the same distance from the boundary. We present a suite of tools for improved data analysis and statistical inference using BD plotting. BD is computed using the Euclidean distance transform after presmoothing and oversampling of nuclear boundaries. Marker distribution profiles are averaged using smoothing with linearly decreasing bandwidth. Average expression curves are scaled and registered by x-axis dilation to compensate for uneven lighting and errors in nuclear boundary marking. Penalized discriminant analysis is used to characterize the quality of separation between average marker distributions. An adaptive piecewise linear model is used to compare expression gradients in intra, peri and extra nuclear zones. The techniques are illustrated by the following: (a) a two sample problem involving a pair of voltage gated calcium channels (Cavl.2 and AB70) marked in different cells; (b) a paired sample problem of calcium channels (Y1F4 and RyR1 ) marked in the same cell.
Universality results for largest eigenvalues of some sample covariance matrix ensembles

For sample covariance matrices with iid entries with sub-Gaussian tails, when both the number of samples and the number of variables become large and the ratio approaches to one, it is a well-known result of A. Soshnikov that the limiting distribution of the largest eigenvalue is same as the of Gaussian samples. In this paper, we extend this result to two cases. The first case is when the ratio approaches to an arbitrary finite value. The second case is when the ratio becomes infinity or arbitrarily small.
Modelling the impact of human activity on nitrogen dioxide concentrations in Europe

Ambient concentrations of many pollutants are associated with emissions due to human activity, such as road transport and other combustion sources. In this paper we consider air pollution as a multi--level phenomenon within a Bayesian hierarchical model. We examine different scales of variation in pollution concentrations ranging from large scale transboundary effects to more localised effects which are directly related to human activity. Specifically, in the first stage of the model, we isolate underlying patterns in pollution concentrations due to global factors such as underlying climate and topography, which are modelled together with spatial structure. At this stage measurements from monitoring sites located within rural areas are used which, as far as possible, are chosen to reflect background concentrations. Having isolated these global effects, in the second stage we assess the effects of human activity on pollution in urban areas. The proposed model was applied to concentrations of nitrogen dioxide measured throughout the EU for which significant increases are found to be associated with human activity in urban areas. The approach proposed here provides valuable information that could be used in performing health impact assessments and to inform policy.
A Hierarchical Bayesian Framework for Constructing Sparsity-inducing Priors

Variable selection techniques have become increasingly popular amongst statisticians due to an increased number of regression and classification applications involving high-dimensional data where we expect some predictors to be unimportant. In this context, Bayesian variable selection techniques involving Markov chain Monte Carlo exploration of the posterior distribution over models can be prohibitively computationally expensive and so there has been attention paid to quasi-Bayesian approaches such as maximum a posteriori (MAP) estimation using priors that induce sparsity in such estimates. We focus on this latter approach, expanding on the hierarchies proposed to date to provide a Bayesian interpretation and generalization of state-of-the-art penalized optimization approaches and providing simultaneously a natural way to include prior information about parameters within this framework. We give examples of how to use this hierarchy to compute MAP estimates for linear and logistic regression as well as sparse precision-matrix estimates in Gaussian graphical models. In addition, an adaptive group lasso method is derived using the framework.
Discussion of: Treelets—An adaptive multi-scale basis for sparse unordered data

DISCUSSION OF: TREELETS—AN ADAPTIVE MULTI-SCALEBASIS FOR SPARSE UNORDERED DATABy Fionn MurtaghUniversity of LondonThe work of Lee et al. is theoretically well founded and thoroughly moti-vated by practical data analysis. The algorithm presented has the followingimportant properties:1. Hierarchical clustering using a novel, adaptive, eigenvector-related, ag-glomerative criterion.2. Principal components analysis carried out locally, leading to the requiredsample size for consistency being logarithmic rather than linear; and com-putational time being quadratic rather than cubic.3. Multiresolution transform with interesting characteristics: data-adaptiveat each node of the tree, orthonormal, and the tree decomposition itselfis data-adaptive.4. Integration of all of the following: hierarchical clustering, dimensionalityreduction, and multiresolution transform.5. Range of data patterns explored, in particular, block patterns in thecovariances, and “model” or pattern contexts.While I admire the work of the authors, nonetheless I have a diﬀerentpoint of view on key aspects of this work:1. The highest dimensionality analyzed seems to be 760 in the Internetadvertisements case study. In fact, the quadratic computational time re-quirements (Section 2.1 of Lee et al.) preclude scalability. My approachin Murtagh (2007a) to wavelet transforming a dendrogram is of linearcomputational complexity (for both observations, and attributes) in themultiresolution transform. The hierarchical clustering, to begin with, istypically quadratic for the n observations, and linear in the p attributes.These computational requirements are necessary for the “small n, largep” problem which motivates this work (Section 1). In particular, linearityin p is a sine qua non for very high dimensionality data exploration.
Optimal spectral norm rates for noisy low-rank matrix completion

In this paper we consider the trace regression model where $n$ entries or linear combinations of entries of an unknown $m_1\times m_2$ matrix $A_0$ corrupted by noise are observed. We establish for the nuclear-norm penalized estimator of $A_0$ introduced in \cite{KLT} a general sharp oracle inequality with the spectral norm for arbitrary values of $n,m_1,m_2$ under an incoherence condition on the sampling distribution $\Pi$ of the observed entries. Then, we apply this method to the matrix completion problem. In this case, we prove that it satisfies an optimal oracle inequality for the spectral norm, thus improving upon the only existing result \cite{KLT} concerning the spectral norm, which assumes that the sampling distribution is uniform. Note that our result is valid, in particular, in the high-dimensional setting $m_1m_2\gg n$. Finally we show that the obtained rate is optimal up to logarithmic factors in a minimax sense.
Sequential design for computer experiments with a flexible Bayesian additive model

In computer experiments, a mathematical model implemented on a computer is used to represent complex physical phenomena. These models, known as computer simulators, enable experimental study of a virtual representation of the complex phenomena. Simulators can be thought of as complex functions that take many inputs and provide an output. Often these simulators are themselves expensive to compute, and may be approximated by “surrogate models” such as statistical regression models. In this paper we consider a new kind of surrogate model, a Bayesian ensemble of trees (Chipman, George, & McCulloch, 2010), with the specific goal of learning enough about the simulator that a particular feature of the simulator can be estimated. We focus on identifying the simulator's global minimum. Utilizing the Bayesian version of the expected improvement criterion (Jones, Schonlau, & Welch, 1998), we show that this ensemble is particularly effective when the simulator is ill‐behaved, exhibiting nonstationarity or abrupt changes in the response. A number of illustrations of the approach are given, including a tidal power application. The Canadian Journal of Statistics 40: 663–678; 2012 © 2012 Statistical Society of Canada
An approximate quantum Cramér–Rao bound based on skew information

A closed-form expression for Wigner-Yanase skew information in mixed-state quantum systems is derived. It is shown that limit values of the mixing coefficients exist such that Wigner-Yanase information is equal to Helstrom information. The latter constitutes an upper bound for the classical expected Fisher information, hence the inverse Wigner-Yanase information provides an approximate lower bound to the variance of an unbiased estimator of the parameter of interest. The advantage of approximating Helstrom's sharp bound lies in the fact that Wigner-Yanase information is straightforward to compute, while it is often very difficult to obtain a feasible expression for Helstrom information. In fact, the latter requires the solution of an implicit second order matrix differential equation, while the former requires just scalar differentiation.
Incompatibility of trends in multi-year estimates from the American Community Survey

The American Community Survey (ACS) provides one-year (ly), three-year (3y) and five-year (5y) multi-year estimates (MYEs) of various demographic and economic variables for each "community," although the ly and 3y may not be available for communities with a small population. These survey estimates are not truly measuring the same quantities, since they each cover different time spans. Using some simplistic models, we demonstrate that comparing different period-length MYEs results in spurious conclusions about trend movements. A simple method utilizing weighted averages is presented that reduces the bias inherent in comparing trends of different MYEs. These weighted averages are nonparametric, require only a short span of data, and are designed to preserve polynomial characteristics of the time series that are relevant for trends. The basic method, which only requires polynomial algebra, is outlined and applied to ACS data. In some cases there is an improvement to comparability, although a final verdict must await additional ACS data. We draw the conclusion that MYE data is not comparable across different periods.
Limits of spiked random matrices II

The top eigenvalues of rank $r$ spiked real Wishart matrices and additively perturbed Gaussian orthogonal ensembles are known to exhibit a phase transition in the large size limit. We show that they have limiting distributions for near-critical perturbations, fully resolving the conjecture of Baik, Ben Arous and Peche [Duke Math. J. (2006) 133 205-235]. The starting point is a new $(2r+1)$-diagonal form that is algebraically natural to the problem; for both models it converges to a certain random Schrodinger operator on the half-line with $r\times r$ matrix-valued potential. The perturbation determines the boundary condition and the low-lying eigenvalues describe the limit, jointly as the perturbation varies in a fixed subspace. We treat the real, complex and quaternion ($\beta=1,2,4$) cases simultaneously. We further characterize the limit laws in terms of a diffusion related to Dyson's Brownian motion, or alternatively a linear parabolic PDE; here $\beta$ appears simply as a parameter. At $\beta=2$, the PDE appears to reconcile with known Painleve formulas for these $r$-parameter deformations of the GUE Tracy-Widom law.
Variable-width confidence intervals in Gaussian regression and penalized maximum likelihood estimators

Hard thresholding, LASSO , adaptive LASSO and SCAD point estimators have been suggested for use in the linear regression context when most of the components of the regression parameter vector are believed to be zero, a sparsity type of assumption. Potscher and Schneider, 2010, Electronic Journal of Statistics, have considered the properties of fixed-width confidence intervals that include one of these point estimators (for all possible data values). They consider a normal linear regression model with orthogonal regressors and show that these confidence intervals are longer than the standard confidence interval (based on the maximum likelihood estimator) when the tuning parameter for these point estimators is chosen to lead to either conservative or consistent model selection. We extend this analysis to the case of variable-width confidence intervals that include one of these point estimators (for all possible data values). In consonance with these findings of Potscher and Schneider, we find that these confidence intervals perform poorly by comparison with the standard confidence interval, when the tuning parameter for these point estimators is chosen to lead to consistent model selection. However, when the tuning parameter for these point estimators is chosen to lead to conservative model selection, our conclusions differ from those of Potscher and Schneider. We consider the variable-width confidence intervals of Farchione and Kabaila, 2008, Statistics & Probability Letters, which have advantages over the standard confidence interval in the context that there is a belief in a sparsity type of assumption. These variable-width confidence intervals are shown to include the hard thresholding, LASSO, adaptive LASSO and SCAD estimators (for all possible data values) provided that the tuning parameters for these estimators are chosen to belong to an appropriate interval.
Irregular sets and Central Limit Theorems for dependent triangular arrays

In previous papers, we studied the asymptotic behaviour of $S_N(A,X)=(2N+1)^{-d/2}\sum_{n \in A_N} X_n,$ where $X$ is a centered, stationary and weakly dependent random field, and $A_N=A \cap [-N,N]^d$, $A \subset \mathbb{Z}^d$. This leads to the definition of asymptotically measurable sets, which enjoy the property that $S_N(A;X)$ has a Gaussian weak limit for any $X$ belonging to a certain class. Here we extend this type of results to the case of weakly dependent triangular arrays and present an application of this technique to regression models. Indeed, we prove that CLT and related results hold for $X_n^N=\varphi(\xi_n^N,Y_n^N), n \in \mathbb{Z}^d$, where $\varphi$ satisfies certain regularity conditions, $\xi$ and $Y$ are independent random fields, $\xi$ is weakly dependent and $Y$ satisfies some Strong Law of Large Numbers.
Berry-Esseen's central limit theorem for non-causal linear processes in Hilbert space

Let $H$ be a real separable Hilbert space and $(a_k)_{k\in\mathbb{Z}}$ a sequence of bounded linear operators from $H$ to $H$. We consider the linear process $X$ defined for any $k$ in $\mathbb{Z}$ by $X_k=\sum_{j\in\mathbb{Z}}a_j(\varepsilon_{k-j})$ where $(\varepsilon_k)_{k\in\mathbb{Z}}$ is a sequence of i.i. d. centered $H$-valued random variables. We investigate the rate of convergence in the CLT for $X$ and in particular we obtain the usual Berry-Esseen's bound provided that $\sum_{j\in\mathbb{Z}}\vert j\vert\|a_j\|_{\mathcal{L}(H)}<+\infty$ and $\varepsilon_0$ belongs to $L_H^{\infty}$.
Sparse Inverse Covariance Estimation via an Adaptive Gradient-Based Method

We study the problem of estimating from data, a sparse approximation to the inverse covariance matrix. Estimating a sparsity constrained inverse covariance matrix is a key component in Gaussian graphical model learning, but one that is numerically very challenging. We address this challenge by developing a new adaptive gradient-based method that carefully combines gradient information with an adaptive step-scaling strategy, which results in a scalable, highly competitive method. Our algorithm, like its predecessors, maximizes an $\ell_1$-norm penalized log-likelihood and has the same per iteration arithmetic complexity as the best methods in its class. Our experiments reveal that our approach outperforms state-of-the-art competitors, often significantly so, for large problems.
ε-Strong simulation of the Brownian path

We present an iterative sampling method which delivers upper and lower bounding processes for the Brownian path. We develop such processes with particular emphasis on being able to unbiasedly simulate them on a personal computer. The dominating processes converge almost surely in the supremum and L 1 norms. In particular, the rate of converge in L 1 is of the order O(K −1/2 ) , K denoting the computing cost. The a.s. enfolding of the Brownian path can be exploited in Monte Carlo applications involving Brownian paths whence our algorithm (termed the e -strong algorithm) can deliver unbiased Monte Carlo estimators over path expectations, overcoming discretisation errors characterising standard approaches. We will show analytical results from applications of the e -strong algorithm for estimating expectations arising in option pricing. We will also illustrate that individual steps of the algorithm can be of separate interest, giving new simulation methods for interesting Brownian distributions.
A Bayesian spatiotemporal model of panel design data: Airborne particle number concentration in Brisbane, Australia

In environmental monitoring, the ability to obtain high‐quality data across space and time is often limited by the cost of purchasing, deploying and maintaining a large collection of equipment, and the employment of personnel to perform these tasks. An ideal design for a monitoring campaign would be dense enough in time to capture short‐range variation at each site, long enough in time to examine trends at each site and across all sites, and dense enough in space to allow modelling of the relationship between the means at each of the sites. This paper outlines a methodology for semiparametric spatiotemporal modelling of data that is dense in time but sparse in space, obtained from a split panel design, the most feasible approach to covering space and time with limited equipment. The data are hourly averaged particle number concentration (PNC) and were collected as part of the International Laboratory for Air Quality and Health's Ultrafine Particles from Traffic Emissions and Children's Health (UPTECH) project. The panel design comprises two weeks of continuous measurements taken at each of a number of government primary schools in the Brisbane Metropolitan Area, with each school visited sequentially. The school data are augmented by data from long‐term monitoring stations at three locations in Brisbane, Australia. The temporal part of the model explains daily and weekly cycles in PNC at the schools. The temporal variation is modelled hierarchically with a penalised random walk term common to all sites and a similar term accounting for the remaining temporal trend at each site. The modelling of temporal trends requires an acknowledgement that the observations are correlated rather than independent. At each school and long‐term monitoring site, peaks in PNC can be attributed to the morning and afternoon rush hour traffic and new particle formation events. The spatial component of the model describes the school‐to‐school variation in mean PNC at each school and within each school ground. The spatial term in the model is derived from a stochastic partial differential equation and approximates a Gaussian process with a Gaussian Markov Random field. Fitting the model helps describe spatial and temporal variability at a subset of the UPTECH schools and the long‐term monitoring sites, which can be used to estimate the exposure of school children to ultrafine particles. Parameter estimates and their uncertainty are computed in a computationally efficient approximate Bayesian inference environment, R‐INLA.