Miracle-dz/UniFAR-FTUs huggingface dataset:
Learning Pairwise Graphical Models with Nonlinear Sufficient Statistics

We investigate a generic problem of learning pairwise exponential family graphical models with pairwise sufficient statistics defined by a global mapping function, e.g., Mercer kernels. This subclass of pairwise graphical models allow us to flexibly capture complex interactions among variables beyond pairwise product. We propose two $\ell_1$-norm penalized maximum likelihood estimators to learn the model parameters from i.i. d. samples. The first one is a joint estimator which estimates all the parameters simultaneously. The second one is a node-wise conditional estimator which estimates the parameters individually for each node. For both estimators, we show that under proper conditions the extra flexibility gained in our model comes at almost no cost of statistical and computational efficiency. We demonstrate the advantages of our model over state-of-the-art methods on synthetic and real datasets.
Algorithms for Envelope Estimation

Envelopes were recently proposed as methods for reducing estimative variation in multivariate linear regression. Estimation of an envelope usually involves optimization over Grassmann manifolds. We propose a fast and widely applicable one-dimensional (1D) algorithm for estimating an envelope in general. We reveal an important structural property of envelopes that facilitates our algorithm, and we prove both Fisher consistency and -consistency of the algorithm. Supplementary materials for this article are available online.
Varying-smoother models for functional responses

This paper studies estimation of a smooth function $f(t,s)$ when we are given functional responses of the form $f(t,\cdot)$ + error, but scientific interest centers on the collection of functions $f(\cdot,s)$ for different $s$. The motivation comes from studies of human brain development, in which $t$ denotes age whereas $s$ refers to brain locations. Analogously to varying-coefficient models, in which the mean response is linear in $t$, the "varying-smoother" models that we consider exhibit nonlinear dependence on $t$ that varies smoothly with $s$. We discuss three approaches to estimating varying-smoother models: (a) methods that employ a tensor product penalty; (b) an approach based on smoothed functional principal component scores; and (c) two-step methods consisting of an initial smooth with respect to $t$ at each $s$, followed by a postprocessing step. For the first approach, we derive an exact expression for a penalty proposed by Wood, and an adaptive penalty that allows smoothness to vary more flexibly with $s$. We also develop "pointwise degrees of freedom," a new tool for studying the complexity of estimates of $f(\cdot,s)$ at each $s$. The three approaches to varying-smoother models are compared in simulations and with a diffusion tensor imaging data set.
Necessary moment conditions for exact reconstruction via basis pursuit

Let $X=(x_1,.,x_n)$ be a random vector that satisfies a weak small ball property and whose coordinates $x_i$ satisfy that $\|x_i\|_{L_p} \lesssim \sqrt{p} \|x_i\|_{L_2}$ for $p \sim \log n$. In \cite{LM_compressed}, it was shown that $N$ independent copies of $X$ can be used as measurement vectors in Compressed Sensing (using the basis pursuit algorithm) to reconstruct any $d$-sparse vector with the optimal number of measurements $N\gtrsim d \log\big(e n/d\big)$. In this note we show that the result is almost optimal. We construct a random vector $X$ with iid, mean-zero, variance one coordinates that satisfies the same weak small ball property and whose coordinates satisfy that $\|x_i\|_{L_p} \lesssim \sqrt{p} \|x_i\|_{L_2}$ for $p \sim (\log n)/(\log N)$, but the basis pursuit algorithm fails to recover even $1$-sparse vectors. The construction shows that `spiky' measurement vectors may lead to a poor performance by the basis pursuit algorithm, but on the other hand may still perform in an optimal way if one chooses a different reconstruction algorithm (like $\ell_0$-minimization). This exhibits the fact that the convex relaxation of $\ell_0$-minimization comes at a significant cost when using `spiky' measurement vectors.
Estimating confidence regions of common measures of (baseline, treatment effect) on dichotomous outcome of a population

In this article we estimate confidence regions of the common measures of (baseline, treatment effect) in observational studies, where the measure of baseline is baseline risk or baseline odds while the measure of treatment effect is odds ratio, risk difference, risk ratio or attributable fraction, and where confounding is controlled in estimation of both baseline and treatment effect. To avoid high complexity of the normal approximation method and the parametric or non-parametric bootstrap method, we obtain confidence regions for measures of (baseline, treatment effect) by generating approximate distributions of the ML estimates of these measures based on one logistic model.
Delete or merge regressors for linear model selection

We consider a problem of linear model selection in the presence of both continuous and categorical predictors. Feasible models consist of subsets of numerical variables and partitions of levels of factors. A new algorithm called delete or merge regressors (DMR) is presented which is a stepwise backward procedure involving ranking the predictors according to squared t-statistics and choosing the final model minimizing BIC. We prove consistency of DMR when the number of predictors tends to infinity with the sample size and describe a simulation study using a pertaining R package. The results indicate significant advantage in time complexity and selection accuracy of our algorithm over Lasso-based methods described in the literature. Moreover, a version of DMR for generalized linear models is
Accounting for Rink Effects in the National Hockey League's Real Time Scoring System

Recording of events in National Hockey League rinks is done through the Real Time Scoring System. This system records events such as hits, shots, faceoffs, etc., as part of the play-by-play files that are made publicly available. Several previous studies have found that there are inconsistencies in the recording of these events from rink to rink. In this paper, we propose a methodology for estimation of the rink effects for each of the rinks in the National Hockey League. Our aim is to build a model which accounts for the relative differences between rinks. We use log-linear regression to model counts of events per game with several predictors including team factors and average score differential. The estimated rink effects can be used to reweight recorded events so that can have comparable counts of events across rinks. Applying our methodology to data from six regular seasons, we find that there are some rinks with rink effects that are significant and consistent across these seasons for multiple events.
Change Points and Temporal Dependence in Reconstructions of Annual Temperature: Did Europe Experience a Little Ice Age?

We analyze the timing and extent of northern European temperature falls during the Little Ice Age, using standard temperature reconstructions. However, we can find little evidence of long swings or structural breaks in European weather before the twentieth century. Instead, European weather between the fifteenth and nineteenth centuries resembles uncorrelated draws from a distribution with a constant mean (although there are decades of markedly lower summer temperature); with the same behaviour holding more tentatively back to the twelfth century. Our results suggest that the existing consensus about a Little Ice Age in Europe may stem from a Slutsky effect, where the standard climatological practice of smoothing data before analysis gives the spurious appearance of irregular oscillations.
Mixtures of Spatial Spline Regressions

We present an extension of the functional data analysis framework for univariate functions to the analysis of surfaces: functions of two variables. The spatial spline regression (SSR) approach developed can be used to model surfaces that are sampled over a rectangular domain. Furthermore, combining SSR with linear mixed effects models (LMM) allows for the analysis of populations of surfaces, and combining the joint SSR-LMM method with finite mixture models allows for the analysis of populations of surfaces with sub-family structures. Through the mixtures of spatial splines regressions (MSSR) approach developed, we present methodologies for clustering surfaces into sub-families, and for performing surface-based discriminant analysis. The effectiveness of our methodologies, as well as the modeling capabilities of the SSR model are assessed through an application to handwritten character recognition.
Classification using log Gaussian Cox processes

McCullagh and Yang (2006) suggest a family of classification algorithms based on Cox processes. We further investigate the log Gaussian variant which has a number of appealing properties. Conditioned on the covariates, the distribution over labels is given by a type of conditional Markov random field. In the supervised case, computation of the predictive probability of a single test point scales linearly with the number of training points and the multiclass generalization is straightforward. We show new links between the supervised method and classical nonparametric methods. We give a detailed analysis of the pairwise graph representable Markov random field, which we use to extend the model to semi-supervised learning problems, and propose an inference method based on graph min-cuts. We give the first experimental analysis on supervised and semi-supervised datasets and show good empirical performance.
Gaussian Approximations and Related Questions for the Spacings process

All the available results on the approximation of the k-spacings process to Gaussian processes have only used one approach, that is the Shorack and Pyke's one. Here, it is shown that this approach cannot yield a rate better than $% \left( N/\log \log N\right) ^{-\frac{1}{4}}\left( \log N\right) ^{\frac{1}{2}% }$. Strong and weak bounds for that rate are specified both where k is fixed and where $k\rightarrow +\infty $. A Glivenko-Cantelli Theorem is given while Stute's result for the increments of the empirical process based on independent and indentically distributed random variables is extended to the spacings process. One of the Mason-Wellner-Shorack cases is also obtained.
The continuum-of-urns scheme, generalized beta and Indian buffet processes, and hierarchies thereof

We describe the combinatorial stochastic process underlying a sequence of conditionally independent Bernoulli processes with a shared beta process hazard measure. As shown by Thibaux and Jordan [TJ07], in the special case when the underlying beta process has a constant concentration function and a finite and nonatomic mean, the combinatorial structure is that of the Indian buffet process (IBP) introduced by Griffiths and Ghahramani [GG05]. By reinterpreting the beta process introduced by Hjort [Hjo90] as a measurable family of Dirichlet processes, we obtain a simple predictive rule for the general case, which can be thought of as a continuum of Blackwell-MacQueen urn schemes (or equivalently, one-parameter Hoppe urn schemes). The corresponding measurable family of Perman-Pitman-Yor processes leads to a continuum of two-parameter Hoppe urn schemes, whose ordinary component is the three-parameter IBP introduced by Teh and G\"or\"ur [TG09], which exhibits power-law behavior, as further studied by Broderick, Jordan, and Pitman [BJP12]. The idea extends to arbitrary measurable families of exchangeable partition probability functions and gives rise to generalizations of the beta process with matching buffet processes. Finally, in the same way that hierarchies of Dirichlet processes were given Chinese restaurant franchise representations by Teh, Jordan, Beal, and Blei [Teh+06], one can construct representations of sequences of Bernoulli processes directed by hierarchies of beta processes (and their generalizations) using the stochastic process we uncover.
A Bayesian Nonparametric Hypothesis Testing Approach for Regression Discontinuity Designs

The regression discontinuity (RD) design is a popular approach to causal inference in non-randomized studies. This is because it can be used to identify and estimate causal effects under mild conditions. Specifically, for each subject, the RD design assigns a treatment or non-treatment, depending on whether or not an observed value of an assignment variable exceeds a fixed and known cutoff value. In this paper, we propose a Bayesian nonparametric regression modeling approach to RD designs, which exploits a local randomization feature. In this approach, the assignment variable is treated as a covariate, and a scalar-valued confounding variable is treated as a dependent variable (which may be a multivariate confounder score). Then, over the model's posterior distribution of locally-randomized subjects that cluster around the cutoff of the assignment variable, inference for causal effects are made within this random cluster, via two-group statistical comparisons of treatment outcomes and non-treatment outcomes. We illustrate the Bayesian nonparametric approach through the analysis of a real educational data set, to investigate the causal link between basic skills and teaching ability.
Exact Bayesian inference in spatiotemporal Cox processes driven by multivariate Gaussian processes

We present a novel inference methodology to perform Bayesian inference for spatiotemporal Cox processes where the intensity function depends on a multivariate Gaussian process. Dynamic Gaussian processes are introduced to enable evolution of the intensity function over discrete time. The novelty of the method lies on the fact that no discretization error is involved despite the non‐tractability of the likelihood function and infinite dimensionality of the problem. The method is based on a Markov chain Monte Carlo algorithm that samples from the joint posterior distribution of the parameters and latent variables of the model. A particular choice of the dominating measure to obtain the likelihood function is shown to be crucial to devise a valid Markov chain Monte Carlo algorithm. The models are defined in a general and flexible way but they are amenable to direct sampling from the relevant distributions because of careful characterization of its components. The models also enable the inclusion of regression covariates and/or temporal components to explain the variability of the intensity function. These components may be subject to relevant interaction with space and/or time. Real and simulated examples illustrate the methodology, followed by concluding remarks.
Covariate Selection in High-Dimensional Generalized Linear Models With Measurement Error

ABSTRACT In many problems involving generalized linear models, the covariates are subject to measurement error. When the number of covariates p exceeds the sample size n, regularized methods like the lasso or Dantzig selector are required. Several recent papers have studied methods which correct for measurement error in the lasso or Dantzig selector for linear models in the p > n setting. We study a correction for generalized linear models, based on Rosenbaum and Tsybakov’s matrix uncertainty selector. By not requiring an estimate of the measurement error covariance matrix, this generalized matrix uncertainty selector has a great practical advantage in problems involving high-dimensional data. We further derive an alternative method based on the lasso, and develop efficient algorithms for both methods. In our simulation studies of logistic and Poisson regression with measurement error, the proposed methods outperform the standard lasso and Dantzig selector with respect to covariate selection, by reducing the number of false positives considerably. We also consider classification of patients on the basis of gene expression data with noisy measurements. Supplementary materials for this article are available online.
Efficient Data Augmentation in Dynamic Models for Binary and Count Data

Dynamic linear models with Gaussian observations and Gaussian states lead to closed-form formulas for posterior simulation. However, these closed-form formulas break down when the response or state evolution ceases to be Gaussian. Dynamic, generalized linear models exemplify a class of models for which this is the case, and include, amongst other models, dynamic binomial logistic regression and dynamic negative binomial regression. Finding and appraising posterior simulation techniques for these models is important since modeling temporally correlated categories or counts is useful in a variety of disciplines, including ecology, economics, epidemiology, medicine, and neuroscience. In this paper, we present one such technique, P\'olya-Gamma data augmentation, and compare it against two competing methods. We find that the P\'olya-Gamma approach works well for dynamic logistic regression and for dynamic negative binomial regression when the count sizes are small. Supplementary files are provided for replicating the benchmarks.
A fast algorithm for sampling from the posterior of a von Mises distribution

Motivated by molecular biology, there has been an upsurge of research activities in directional statistics in general and its Bayesian aspect in particular. The central distribution for the circular case is von Mises distribution which has two parameters (mean and concentration) akin to the univariate normal distribution. However, there has been a challenge to sample efficiently from the posterior distribution of the concentration parameter. We describe a novel, highly efficient algorithm to sample from the posterior distribution and fill this long-standing gap.
Elastic Net Procedure for Partially Linear Models

Variable selection plays an important role in the high-dimensional data analysis. However the high-dimensional data often induces the strongly correlated variables problem. In this paper, we propose Elastic Net procedure for partially linear models and prove the group effect of its estimate. By a simulation study, we show that the strongly correlated variables problem can be better handled by the Elastic Net procedure than Lasso, ALasso and Ridge. Based on an empirical analysis, we can get that the Elastic Net procedure is particularly useful when the number of predictors $p$ is much bigger than the sample size $n$.
Bayesian inference of Gaussian mixture models with noninformative priors

This paper deals with Bayesian inference of a mixture of Gaussian distributions. A novel formulation of the mixture model is introduced, which includes the prior constraint that each Gaussian component is always assigned a minimal number of data points. This enables noninformative improper priors such as the Jereys prior to be placed on the component parameters. We demonstrate diculties involved in specifying a prior for the standard Gaussian mixture model, and show how the new model can be used to overcome these. MCMC methods are given for ecient sampling from the posterior of this model.
Functional clustering in nested designs: Modeling variability in reproductive epidemiology studies

We discuss functional clustering procedures for nested designs, where multiple curves are collected for each subject in the study. We start by considering the application of standard functional clustering tools to this problem, which leads to groupings based on the average profile for each subject. After discussing some of the shortcomings of this approach, we present a mixture model based on a generalization of the nested Dirichlet process that clusters subjects based on the distribution of their curves. By using mixtures of generalized Dirichlet processes, the model induces a much more flexible prior on the partition structure than other popular model-based clustering methods, allowing for different rates of introduction of new clusters as the number of observations increases. The methods are illustrated using hormone profiles from multiple menstrual cycles collected for women in the Early Pregnancy Study.
Relabelling Algorithms for Large Dataset Mixture Models

Mixture models are flexible tools in density estimation and classification problems. Bayesian estimation of such models typically relies on sampling from the posterior distribution using Markov chain Monte Carlo. Label switching arises because the posterior is invariant to permutations of the component parameters. Methods for dealing with label switching have been studied fairly extensively in the literature, with the most popular approaches being those based on loss functions. However, many of these algorithms turn out to be too slow in practice, and can be infeasible as the size and dimension of the data grow. In this article, we review earlier solutions which can scale up well for large data sets, and compare their performances on simulated and real datasets. In addition, we propose a new, and computationally efficient algorithm based on a loss function interpretation, and show that it can scale up well in larger problems. We conclude with some discussions and recommendations of all the methods studied.
Comparing and Forecasting Performances in Different Events of Athletics Using a Probabilistic Model

Though athletics statistics are abundant, it is a difficult task to quantitatively compare performances from different events of track, field, and road running in a meaningful way. There are several commonly-used methods, but each has its limitations. Some methods, for example, are valid only for running events, or are unable to compare men's performances to women's, while others are based largely on world records and are thus unsuitable for comparing world records to one other. The most versatile and widely-used statistic is a set of scoring tables compiled by the IAAF, which are updated and published every few years. Unfortunately, these methods are not fully disclosed. In this paper, we propose a straight-forward, objective, model-based algorithm for assigning scores to athletic performances for the express purpose of comparing marks between different events. Specifically, the main score we propose is based on the expected number of athletes who perform better than a given mark within a calendar year. Computing this naturally interpretable statistic requires only a list of the top performances in each event and is not overly dependent on a small number of marks, such as the world records. We found that this statistic could predict the quality of future performances better than the IAAF scoring tables, and is thus better suited for comparing performances from different events. In addition, the probabilistic model used to generate the performance scores allows for multiple interpretations which can be adapted for various purposes, such as calculating the expected top mark in a given event or calculating the probability of a world record being broken within a certain time period. In this paper, we give the details of the model and the scores, a comparison with the IAAF scoring tables, and a demonstration of how we can calculate expectations of what might happen in the coming Olympic year. Our conclusion is that a probabilistic model such as the one presented here is a more informative and more versatile choice than the standard methods for comparing athletic performances.