Miracle-dz/UniFAR-FTUs huggingface dataset:
Trust, but verify: benefits and pitfalls of least-squares refitting in high dimensions

Least-squares refitting is widely used in high dimensional regression to reduce the prediction bias of l1-penalized estimators (e.g., Lasso and Square-Root Lasso). We present theoretical and numerical results that provide new insights into the benefits and pitfalls of least-squares refitting. In particular, we consider both prediction and estimation, and we pay close attention to the effects of correlations in the design matrices of linear regression models, since these correlations - although often neglected - are crucial in the context of linear regression, especially in high dimensional contexts. First, we demonstrate that the benefit of least-squares refitting strongly depends on the setting and task under consideration: least-squares refitting can be beneficial even for settings with highly correlated design matrices but is not advisable for all settings, and least-squares refitting can be beneficial for estimation but performs better for prediction. Finally, we introduce a criterion that indicates whether least-squares refitting is advisable for a specific setting and task under consideration, and we conduct a thorough simulation study involving the Lasso to show the usefulness of this criterion.
On pattern recovery of the fused Lasso

We study the property of the Fused Lasso Signal Approximator (FLSA) for estimating a blocky signal sequence with additive noise. We transform the FLSA to an ordinary Lasso problem. By studying the property of the design matrix in the transformed Lasso problem, we find that the irrepresentable condition might not hold, in which case we show that the FLSA might not be able to recover the signal pattern. We then apply the newly developed preconditioning method -- Puffer Transformation [Jia and Rohe, 2012] on the transformed Lasso problem. We call the new method the preconditioned fused Lasso and we give non-asymptotic results for this method. Results show that when the signal jump strength (signal difference between two neighboring groups) is big and the noise level is small, our preconditioned fused Lasso estimator gives the correct pattern with high probability. Theoretical results give insight on what controls the signal pattern recovery ability -- it is the noise level {instead of} the length of the sequence. Simulations confirm our theorems and show significant improvement of the preconditioned fused Lasso estimator over the vanilla FLSA.
Adaptive-treed bandits

We describe a novel algorithm for noisy global optimisation and continuum-armed bandits, with good convergence properties over any continuous reward function having finitely many polynomial maxima. Over such functions, our algorithm achieves square-root regret in bandits, and inverse-square-root error in optimisation, without prior information. Our algorithm works by reducing these problems to tree-armed bandits, and we also provide new results in this setting. We show it is possible to adaptively combine multiple trees so as to minimise the regret, and also give near-matching lower bounds on the regret in terms of the zooming dimension.
Laminations from the Main Cubioid

According to a recent paper \cite{bopt13}, polynomials from the closure $\bar{\rm PHD}_3$ of the {\em Principal Hyperbolic Domain} ${\rm PHD}_3$ of the cubic connectedness locus have a few specific properties. The family $\mathrm{CU}$ of all polynomials with these properties is called the \emph{Main Cubioid}. In this paper we describe the set $\mathrm{CU}^c$ of laminations which can be associated to polynomials from $\mathrm{CU}$.
SMOOTHED QUANTILE REGRESSION PROCESSES FOR BINARY RESPONSE MODELS

In this article, we consider binary response models with linear quantile restrictions. Considerably generalizing previous research on this topic, our analysis focuses on an infinite collection of quantile estimators. We derive a uniform linearization for the properly standardized empirical quantile process and discover some surprising differences with the setting of continuously observed responses. Moreover, we show that considering quantile processes provides an effective way of estimating binary choice probabilities without restrictive assumptions on the form of the link function, heteroskedasticity, or the need for high dimensional nonparametric smoothing necessary for approaches available so far. A uniform linear representation and results on asymptotic normality are provided, and the connection to rearrangements is discussed.
A code arithmetic approach for quaternary code designs and its application to (1/64)th-fractions

The study of good nonregular fractional factorial designs has received significant attention over the last two decades. Recent research indicates that designs constructed from quaternary codes (QC) are very promising in this regard. The present paper aims at exploring the fundamental structure and developing a theory to characterize the wordlengths and aliasing indexes for a general $(1/4)^p$th-fraction QC design. Then the theory is applied to (1/64)th-fraction QC designs. Examples are given, indicating that there exist some QC designs that have better design properties, and are thus more cost-efficient, than the regular fractional factorial designs of the same size. In addition, a result about the periodic structure of (1/64)th-fraction QC designs regarding resolution is stated.
Covariance Inflation in the Ensemble Kalman Filter: A Residual Nudging Perspective and Some Implications

AbstractThis article examines the influence of covariance inflation on the distance between the measured observation and the simulated (or predicted) observation with respect to the state estimate. In order for the aforementioned distance to be bounded in a certain interval, some sufficient conditions are derived, indicating that the covariance inflation factor should be bounded in a certain interval, and that the inflation bounds are related to the maximum and minimum eigenvalues of certain matrices. Implications of these analytic results are discussed, and a numerical experiment is presented to verify the validity of the analysis conducted.
Statistical inference on transformation models: a self-induced smoothing approach

ABSTRACT This paper deals with a general class of transformation models that contains many important semiparametric regression models as special cases. It develops a self-induced smoothing for the maximum rank correlation estimator, resulting in simultaneous point and variance estimation. The self-induced smoothing does not require bandwidth selection, yet provides the right amount of smoothness so that the estimator is asymptotically normal with mean zero (unbiased) and variance–covariance matrix consistently estimated by the usual sandwich-type estimator. An iterative algorithm is given for the variance estimation and shown to numerically converge to a consistent limiting variance estimator. The approach is applied to a data set involving survival times of primary biliary cirrhosis patients. Simulation results are reported, showing that the new method performs well under a variety of scenarios.
Combining P-Values Via Averaging

This paper proposes general methods for the problem of multiple testing of a single hypothesis, with a standard goal of combining a number of $p$-values without making any assumptions about their dependence structure. A result by Rüschendorf (1982) and, independently, Meng (1993) implies that the $p$-values can be combined by scaling up their arithmetic mean by a factor of 2, and no smaller factor is sufficient in general. A similar result by Mattner about the geometric mean replaces 2 by e. Based on more recent developments in mathematical finance, specifically, robust risk aggregation techniques, we extend these results to generalized means; in particular, we show that $K$ $p$-values can be combined by scaling up their harmonic mean by a factor of $\log K$ asymptotically as $K$ tends to infinity. This leads to a generalized version of the Bonferroni–Holm procedure. We also explore methods using weighted averages of $p$-values. Finally, we discuss the efficiency of various methods of combining $p$-values and how to choose a suitable method in light of data and prior information.
Trends in storm surge probability of occurrence along the Polish Baltic Sea coast

When assessing the hazard brought by storm surges on the coast, a frequency analysis is often conducted. An issue of particular concern is the sea level rise (SLR), thought to have an impact on maximum water levels. In this study, three gauge stations with long records dating back to the 19th century and located along the Polish Baltic Sea coast were analyzed. Different theoretical distributions were tested in order to find the best-fitting one. Gumbel distribution was chosen and estimated for 40-year moving periods. The location parameter soared in all tide gauges, but the shape parameter was stable in Gdansk and decreased in the other two. In effect, theoretical annual maximum water levels followed the SLR in Gdansk, whereas most extreme storm surges had no trend in Swinoujscie and a negative one in Kolobrzeg. A possible explanation for this was investigated (change in atmospheric circulation measured by NAO index).
Asymptotic properties of the partition function and applications in tail index inference of heavy-tailed data

The so-called partition function is a sample moment statistic based on blocks of data and it is often used in the context of multifractal processes. It will be shown that its behaviour is strongly influenced by the tail of the distribution underlying the data both in independent identically distributed and weakly dependent cases. These results will be used to develop graphical and estimation methods for the tail index of a distribution. The performance of the tools proposed is analysed and compared with other methods by means of simulations and examples.
Testing the regularity of a smooth signal

We develop a test to determine whether a function lying in a fixed $L_2$-Sobolev-type ball of smoothness $t$, and generating a noisy signal, is in fact of a given smoothness $s\geq t$ or not. While it is impossible to construct a uniformly consistent test for this problem on every function of smoothness $t$, it becomes possible if we remove a sufficiently large region of the set of functions of smoothness $t$. The functions that we remove are functions of smoothness strictly smaller than $s$, but that are very close to $s$-smooth functions. A lower bound on the size of this region has been proved to be of order $n^{-t/(2t+1/2)}$, and in this paper, we provide a test that is consistent after the removal of a region of such a size. Even though the null hypothesis is composite, the size of the region we remove does not depend on the complexity of the null hypothesis.
Multiresolution Analysis of Incomplete Rankings

Incomplete rankings on a set of items $\{1,\; \ldots,\; n\}$ are orderings of the form $a_{1}\prec\dots\prec a_{k}$, with $\{a_{1},\dots a_{k}\}\subset\{1,\dots,n\}$ and $k < n$. Though they arise in many modern applications, only a few methods have been introduced to manipulate them, most of them consisting in representing any incomplete ranking by the set of all its possible linear extensions on $\{1,\; \ldots,\; n\}$. It is the major purpose of this paper to introduce a completely novel approach, which allows to treat incomplete rankings directly, representing them as injective words over $\{1,\; \ldots,\; n\}$. Unexpectedly, operations on incomplete rankings have very simple equivalents in this setting and the topological structure of the complex of injective words can be interpretated in a simple fashion from the perspective of ranking. We exploit this connection here and use recent results from algebraic topology to construct a multiresolution analysis and develop a wavelet framework for incomplete rankings. Though purely combinatorial, this construction relies on the same ideas underlying multiresolution analysis on a Euclidean space, and permits to localize the information related to rankings on each subset of items. It can be viewed as a crucial step toward nonlinear approximation of distributions of incomplete rankings and paves the way for many statistical applications, including preference data analysis and the design of recommender systems.
Comparison of non-homogeneous regression models for probabilistic wind speed forecasting

In weather forecasting, non-homogeneous regression (NR) is used to statistically post-process forecast ensembles in order to obtain calibrated predictive distributions. For wind speed forecasts, the regression model is given by a truncated normal (TN) distribution, where location and spread derive from the ensemble. This article proposes two alternative approaches which utilise the generalised extreme value (GEV) distribution. A direct alternative to the TN regression is to apply a predictive distribution from the GEV family, while a regime-switching approach based on the median of the forecast ensemble incorporates both distributions. In a case study on daily maximum wind speed over Germany with the forecast ensemble from the European Centre for Medium-Range Weather Forecasts (ECMWF), all three approaches significantly improve the calibration as well as the overall skill of the raw ensemble with the regime-switching approach showing the highest skill in the upper tail.
Adaptive function estimation in nonparametric regression with one-sided errors

We consider the model of nonregular nonparametric regression where smoothness constraints are imposed on the regression function $f$ and the regression errors are assumed to decay with some sharpness level at their endpoints. The aim of this paper is to construct an adaptive estimator for the regression function $f$. In contrast to the standard model where local averaging is fruitful, the nonregular conditions require a substantial different treatment based on local extreme values. We study this model under the realistic setting in which both the smoothness degree $\beta>0$ and the sharpness degree $\mathfrak {a}\in(0,\infty)$ are unknown in advance. We construct adaptation procedures applying a nested version of Lepski's method and the negative Hill estimator which show no loss in the convergence rates with respect to the general $L_q$-risk and a logarithmic loss with respect to the pointwise risk. Optimality of these rates is proved for $\mathfrak{a}\in(0,\infty)$. Some numerical simulations and an application to real data are provided.
Theoretical grounding for estimation in conditional independence multivariate finite mixture models

ABSTRACT For the nonparametric estimation of multivariate finite mixture models with the conditional independence assumption, we propose a new formulation of the objective function in terms of penalised smoothed Kullback–Leibler distance. The nonlinearly smoothed majorisation-minimisation (NSMM) algorithm is derived from this perspective. An elegant representation of the NSMM algorithm is obtained using a novel projection-multiplication operator, a more precise monotonicity property of the algorithm is discovered, and the existence of a solution to the main optimisation problem is proved for the first time.
Solution of linear ill-posed problems using overcomplete dictionaries

In the present paper we consider application of overcomplete dictionaries to solution of general ill-posed linear inverse problems. Construction of an adaptive optimal solution for such problems usually relies either on a singular value decomposition or representation of the solution via an orthonormal basis. The shortcoming of both approaches lies in the fact that, in many situations, neither the eigenbasis of the linear operator nor a standard orthonormal basis constitutes an appropriate collection of functions for sparse representation of the unknown function. In the context of regression problems, there have been an enormous amount of effort to recover an unknown function using an overcomplete dictionary. One of the most popular methods, Lasso, is based on minimizing the empirical likelihood and requires stringent assumptions on the dictionary, the, so called, compatibility conditions. While these conditions may be satisfied for the original dictionary functions, they usually do not hold for their images due to contraction imposed by the linear operator. In what follows, we bypass this difficulty by a novel approach which is based on inverting each of the dictionary functions and matching the resulting expansion to the true function, thus, avoiding unrealistic assumptions on the dictionary and using Lasso in a predictive setting. We examine both the white noise and the observational model formulations and also discuss how exact inverse images of the dictionary functions can be replaced by their approximate counterparts. Furthermore, we show how the suggested methodology can be extended to the problem of estimation of a mixing density in a continuous mixture. For all the situations listed above, we provide the oracle inequalities for the risk in a finite sample setting. Simulation studies confirm good computational properties of the Lasso-based technique.
Variable selection and estimation for semi-parametric multiple-index models

In this paper, we propose a novel method to select significant variables and estimate the corresponding coefficients in multiple-index models with a group structure. All existing approaches for single-index models cannot be extended directly to handle this issue with several indices. This method integrates a popularly used shrinkage penalty such as LASSO with the group-wise minimum average variance estimation. It is capable of simultaneous dimension reduction and variable selection, while incorporating the group structure in predictors. Interestingly, the proposed estimator with the LASSO penalty then behaves like an estimator with an adaptive LASSO penalty. The estimator achieves consistency of variable selection without sacrificing the root-$n$ consistency of basis estimation. Simulation studies and a real-data example illustrate the effectiveness and efficiency of the new method.
The accessible lasso models

ABSTRACT A new line of research [Lee et al. Exact post-selection inference with the lasso. arXiv preprint arXiv:1311.6238 (2013)] on the lasso [Tibshirani. Regression shrinkage and selection via the lasso. J R Stat Soc Ser B (Methodological). 1996;267–288] exploits the beautiful geometric fact that the lasso fit is the residual from projecting the response vector y onto a certain convex polytope [Tibshirani et al. Degrees of freedom in lasso problems. Ann Stat. 2012;40:1198–1232]. This geometric picture also allows an exact geometric description of the set of accessible lasso models for a given design matrix, that is, which configurations of the signs of the coefficients it is possible to realize with some choice of y. In particular, the accessible lasso models are those that correspond to a face of the convex hull of all the feature vectors together with their negations. This convex hull representation then permits the enumeration and bounding of the number of accessible lasso models, which in turn provides a direct proof of model selection inconsistency when the size of the true model is greater than half the number of observations.
Convergence rates of maximal deviation distribution for projection estimates of Levy densities

In this paper, we consider projection estimates for Levy densi- ties in high-frequency setup. We give a unified treatment for different sets of basis functions and focus on the asymptotic properties of the maximal deviation distribution for these estimates. Our results are based on the idea to reformulate the problems in terms of Gaussian processes of some special type and to further analyze these Gaussian processes. In particular, we con- struct a sequence of excursion sets, which guarantees the convergence of the maximal deviation distribution to the Gumbel distribution. We show that the rates of convergence presented in previous research on this topic are logarithmic and construct the sequences of accompanying laws with power rates. MSC 2010 subject classifications: Primary 60G51, 62M99; secondary 62G05.
Optimal stopping problems for a Brownian motion with a disorder on a finite interval

We consider optimal stopping problems for a Brownian motion and a geometric Brownian motion with a "disorder", assuming that the moment of a disorder is uniformly distributed on a finite interval. Optimal stopping rules are found as the first hitting times of some Markov process (the Shiryaev-Roberts statistic) to time-dependent boundaries, which are characterized by certain Volterra integral equations. The problems considered are related to mathematical finance and can be applied in questions of choosing the optimal time to sell an asset with changing trend.
Empirical Bayes posterior concentration in sparse high-dimensional linear models

We propose a new empirical Bayes approach for inference in the $p \gg n$ normal linear model. The novelty is the use of data in the prior in two ways, for centering and regularization. Under suitable sparsity assumptions, we establish a variety of concentration rate results for the empirical Bayes posterior distribution, relevant for both estimation and model selection. Computation is straightforward and fast, and simulation results demonstrate the strong finite-sample performance of the empirical Bayes model selection procedure.
Consistency of the Adaptive Multiple Importance Sampling

Among Monte Carlo techniques, the importance sampling requires fine tuning of a proposal distribution, which is now fluently resolved through iterative schemes. The Adaptive Multiple Importance Sampling (AMIS) of Cornuet et al. (2012) provides a significant improvement in stability and effective sample size due to the introduction of a recycling procedure. However, the consistency of the AMIS estimator remains largely open. In this work we prove the convergence of the AMIS, at a cost of a slight modification in the learning process. Contrary to Douc et al. (2007a), results are obtained here in the asymptotic regime where the number of iterations is going to infinity while the number of drawings per iteration is a fixed, but growing sequence of integers. Hence some of the results shed new light on adaptive population Monte Carlo algorithms in that last regime.
Penalising Model Component Complexity: A Principled, Practical Approach to Constructing Priors

In this paper, we introduce a new concept for constructing prior distributions. We exploit the natural nested structure inherent to many model components, which defines the model component to be a flexible extension of a base model. Proper priors are defined to penalise the complexity induced by deviating from the simpler base model and are formulated after the input of a user-defined scaling parameter for that model component, both in the univariate and the multivariate case. These priors are invariant to reparameterisations, have a natural connection to Jeffreys’ priors, are designed to support Occam’s razor and seem to have excellent robustness properties, all which are highly desirable and allow us to use this approach to define default prior distributions. Through examples and theoretical results, we demonstrate the appropriateness of this approach and how it can be applied in various situations.