Miracle-dz/UniFAR-FTUs huggingface dataset:
Estimation of low-rank tensors via convex optimization

In this paper, we propose three approaches for the estimation of the Tucker decomposition of multi-way arrays (tensors) from partial observations. All approaches are formulated as convex minimization problems. Therefore, the minimum is guaranteed to be unique. The proposed approaches can automatically estimate the number of factors (rank) through the optimization. Thus, there is no need to specify the rank beforehand. The key technique we employ is the trace norm regularization, which is a popular approach for the estimation of low-rank matrices. In addition, we propose a simple heuristic to improve the interpretability of the obtained factorization. The advantages and disadvantages of three proposed approaches are demonstrated through numerical experiments on both synthetic and real world datasets. We show that the proposed convex optimization based approaches are more accurate in predictive performance, faster, and more reliable in recovering a known multilinear structure than conventional approaches.
The stochastic system approach to causality with a view toward lifecourse epidemiology

The approach of causality based on physical laws and systems is revisited. The issue of "levels", the relevance to epidemiology and the definition of effects are particularly developed. Moreover it is argued that this approach that we call the stochastic system approach is particularly well fitted to study lifecourse epidemiology. A hierarchy of factors is described that could be modeled using a suitable multivariate stochastic process. To illustrate this approach, a conceptual model for coronary heart disease mixing continuous and discrete state-space processes is proposed.
Conditional simulation of extremal Gaussian processes

Recently the regular conditional distributions of max-infinitely divisible processes were derived by \citet{Dombry2011} and although these conditional distributions have complicated closed forms, \citet{Dombry2011b} introduce an algorithm to get conditional realizations of Brown-Resnick processes. In this paper we derive the regular conditional distributions of the max-stable process introduced by \citet{Schlather2002} and adapt the framework of \citet{Dombry2011b} to this specific process. We test the methods on simulated data and give an application to extreme temperatures in Switzerland. Results show that the proposed sampling scheme provide accurate conditional simulations and can handle real-sized problems.
Bayes factors and the geometry of discrete hierarchical loglinear models

A standard tool for model selection in a Bayesian framework is the Bayes factor which compares the marginal likelihood of the data under two given different models. In this paper, we consider the class of hierarchical loglinear models for discrete data given under the form of a contingency table with multinomial sampling. We assume that the Diaconis-Ylvisaker conjugate prior is the prior distribution on the loglinear parameters and the uniform is the prior distribution on the space of models. Under these conditions, the Bayes factor between two models is a function of their prior and posterior normalizing constants. These constants are functions of the hyperparameters $(m,\alpha)$ which can be interpreted respectively as marginal counts and the total count of a fictive contingency table. We study the behaviour of the Bayes factor when $\alpha$ tends to zero. In this study two mathematical objects play a most important role. They are, first, the interior $C$ of the convex hull $\bar{C}$ of the support of the multinomial distribution for a given hierarchical loglinear model together with its faces and second, the characteristic function $\mathbb{J}_C$ of this convex set $C$. We show that, when $\alpha$ tends to 0, if the data lies on a face $F_i$ of $\bar{C_i},i=1,2$ of dimension $k_i$, the Bayes factor behaves like $\alpha^{k_1-k_2}$. This implies in particular that when the data is in $C_1$ and in $C_2$, i.e. when $k_i$ equals the dimension of model $J_i$, the sparser model is favored, thus confirming the idea of Bayesian regularization.
Mixture model for designs in high dimensional regression and the LASSO

The LASSO is a recent technique for variable selection in the regression model \bean y&=&X\beta + z, \eean where $X\in \R^{n\times p}$ and $z$ is a centered gaussian i.i. d. noise vector $\mathcal N(0,\sigma^2I)$. The LASSO has been proved to achieve remarkable properties such as exact support recovery of sparse vectors when the columns are sufficently incoherent and low prediction error under even less stringent conditions. However, many matrices do not satisfy small coherence in practical applications and the LASSO estimator may thus suffer from what is known as the slow rate regime. The goal of the present paper is to study the LASSO from a slightly different perspective by proposing a mixture model for the design matrix which is able to capture in a natural way the potentially clustered nature of the columns in many practical situations. In this model, the columns of the design matrix are drawn from a Gaussian mixture model. Instead of requiring incoherence for the design matrix $X$, we only require incoherence of the much smaller matrix of the mixture's centers. Our main result states that $X\beta$ can be estimated with the same precision as for incoherent designs except for a correction term depending on the maximal variance in the mixture model.
Graph Construction for Learning with Unbalanced Data

Unbalanced data arises in many learning tasks such as clustering of multi-class data, hierarchical divisive clustering and semisupervised learning. Graph-based approaches are popular tools for these problems. Graph construction is an important aspect of graph-based learning. We show that graph-based algorithms can fail for unbalanced data for many popular graphs such as k-NN, \epsilon-neighborhood and full-RBF graphs. We propose a novel graph construction technique that encodes global statistical information into node degrees through a ranking scheme. The rank of a data sample is an estimate of its p-value and is proportional to the total number of data samples with smaller density. This ranking scheme serves as a surrogate for density; can be reliably estimated; and indicates whether a data sample is close to valleys/modes. This rank-modulated degree(RMD) scheme is able to significantly sparsify the graph near valleys and provides an adaptive way to cope with unbalanced data. We then theoretically justify our method through limit cut analysis. Unsupervised and semi-supervised experiments on synthetic and real data sets demonstrate the superiority of our method.
On the Hermite spline conjecture and its connection to k-monotone densities

The k-monotone classes of densities defined on (0, \infty) have been known in the mathematical literature but were for the first time considered from a statistical point of view by Balabdaoui and Wellner (2007, 2010). In these works, the authors generalized the results established for monotone (k=1) and convex (k=2) densities by giving a characterization of the Maximum Likelihood and Least Square estimators (MLE and LSE) and deriving minimax bounds for rates of convergence. For k strictly larger than 2, the pointwise asymptotic behavior of the MLE and LSE studied by Balabdaoui and Wellner (2007) would show that the MLE and LSE attain the minimax lower bounds in a local pointwise sense. However, the theory assumes that a certain conjecture about the approximation error of a Hermite spline holds true. The main goal of the present note is to show why such a conjecture cannot be true. We also suggest how to bypass the conjecture and rebuild the key proofs in the limit theory of the estimators.
Semi-parametric regression: Efficiency gains from modeling the nonparametric part

It is widely admitted that structured nonparametric modeling that circumvents the curse of dimensionality is important in nonparametric estimation. In this paper we show that the same holds for semi-parametric estimation. We argue that estimation of the parametric component of a semi-parametric model can be improved essentially when more structure is put into the nonparametric part of the model. We illustrate this for the partially linear model, and investigate efficiency gains when the nonparametric part of the modelhas an additive structure. We present the semi-parametric Fisher information bound for estimating the parametric part of the partially linear additive model and provide semi-parametric efficient estimators for which we use a smooth backfitting technique to deal with the additive nonparametric part. We also present the finite sample performances of the proposed estimators and analyze Boston housing data as an illustration.
Local linear estimator for stochastic differential equations driven by $\alpha$-stable L\'{e}vy motions

We study the local linear estimator for the drift coefficient of stochastic differential equations driven by $\alpha$-stable L\'{e}vy motions observed at discrete instants letting $T \rightarrow \infty$. Under regular conditions, we derive the weak consistency and central limit theorem of the estimator. Compare with Nadaraya-Watson estimator, the local linear estimator has a bias reduction whether kernel function is symmetric or not under different schemes.
Informant Discrepancies and the Heritability of Antisocial Behavior: A Meta-Analysis

Antisocial behavior, which includes both aggressive and delinquent activities, is the opposite of prosocial behavior. Researchers have studied the heritability of antisocial behavior among twin and non-twin sibling pairs from behavioral ratings made by parents, teachers, observers, and youth. Through a meta-analysis, we examined longitudinal and cross sectional research in the behavioral genetics of antisocial behavior, consisting of 42 studies, of which 38 were studies of twin pairs, 3 were studies of twins and non-twin siblings, and 1 was a study of adoptees. These studies provided n = 89 heritability (h2) effect size estimates from a total of 94,517 sibling pairs who ranged in age from 1.5 to 18 years; studies provided data for 29 moderators (predictors). We employed a random-effects meta-analysis model to achieve three goals: (a) perform statistical inference of the overall heritability distribution in the underlying population of studies, (b) identify significant study level moderators (predictors) of heritability, and (c) examine how the heritability distribution varied as a function of age and type of informant, particularly in longitudinal research. The meta-analysis indicated a bimodal overall heritability distribution, indicating two clusters of moderate and high heritability values, respectively; identified four moderators that predicted significant changes in mean heritability; and indicated differential patterns of median h2 and variance (interquartile ranges) across informants and ages. We argue for a cross-perspective, cross-setting model for selecting informants in behavioral genetic research, that is flexible and sensitive to changes in antisocial behavior over time.
A Bayesian Boosting Model

We offer a novel view of AdaBoost in a statistical setting. We propose a Bayesian model for binary classification in which label noise is modeled hierarchically. Using variational inference to optimize a dynamic evidence lower bound, we derive a new boosting-like algorithm called VIBoost. We show its close connections to AdaBoost and give experimental results from four datasets.
Constraints on Yield Parameters in Extended Maximum Likelihood Fits

The method of extended maximum likelihood is a well known concept of parameter estimation. One can implement external knowledge on the unknown parameters by multiplying the likelihood by constraint terms. In this note, we emphasize that this is also true for yield parameters in an extended maximum likelihood fit, which is widely used in the particle physics community. We recommend a way to generate pseudo-experiments in presence of constraint terms on yield parameters, and point to pitfalls inside the RooFit framework.
The EFM approach for single-index models

Single-index models are natural extensions of linear models and circumvent the so-called curse of dimensionality. They are becoming increasingly popular in many scientific fields including biostatistics, medicine, economics and financial econometrics. Estimating and testing the model index coefficients $\bolds{\beta}$ is one of the most important objectives in the statistical analysis. However, the commonly used assumption on the index coefficients, $\|\bolds{\beta}\|=1$, represents a nonregular problem: the true index is on the boundary of the unit ball. In this paper we introduce the EFM approach, a method of estimating functions, to study the single-index model. The procedure is to first relax the equality constraint to one with (d-1) components of $\bolds{\beta}$ lying in an open unit ball, and then to construct the associated (d-1) estimating functions by projecting the score function to the linear space spanned by the residuals with the unknown link being estimated by kernel estimating functions. The root-n consistency and asymptotic normality for the estimator obtained from solving the resulting estimating equations are achieved, and a Wilks type theorem for testing the index is demonstrated. A noticeable result we obtain is that our estimator for $\bolds{\beta}$ has smaller or equal limiting variance than the estimator of Carroll et al. [J. Amer. Statist. Assoc. 92 (1997) 447-489]. A fixed-point iterative scheme for computing this estimator is proposed. This algorithm only involves one-dimensional nonparametric smoothers, thereby avoiding the data sparsity problem caused by high model dimensionality. Numerical studies based on simulation and on applications suggest that this new estimating system is quite powerful and easy to implement.
A New Monte Carlo Based Algorithm for the Gaussian Process Classification Problem

Gaussian process is a very promising novel technology that has been applied to both the regression problem and the classification problem. While for the regression problem it yields simple exact solutions, this is not the case for the classification problem, because we encounter intractable integrals. In this paper we develop a new derivation that transforms the problem into that of evaluating the ratio of multivariate Gaussian orthant integrals. Moreover, we develop a new Monte Carlo procedure that evaluates these integrals. It is based on some aspects of bootstrap sampling and acceptancerejection. The proposed approach has beneficial properties compared to the existing Markov Chain Monte Carlo approach, such as simplicity, reliability, and speed.
High-dimensional Feature Selection Using Hierarchical Bayesian Logistic Regression with Heavy-tailed Priors

The problem of selecting the most useful features from a great many (eg, thousands) of candidates arises in many areas of modern sciences. An interesting problem from genomic research is that, from thousands of genes that are active (expressed) in certain tissue cells, we want to find the genes that can be used to separate tissues of different classes (eg. cancer and normal). In this paper, we report our empirical experiences of using Bayesian logistic regression based on heavy-tailed priors with moderately small degree freedom (such as 1) and very small scale, and using Hamiltonian Monte Carlo to do computation. We discuss the advantages and limitations of this method, and illustrate the difficulties that remain unsolved. The method is applied to a real microarray data set related to prostate cancer. The method identifies only 3 non-redundant genes out of 6033 candidates but achieves better leave-one-out cross-validated prediction accuracy than many other methods.
Kuramoto model with frequency-degree correlations on complex networks

We study the Kuramoto model on complex networks, in which natural frequencies of phase oscillators and the vertex degrees are correlated. Using the annealed network approximation and numerical simulations, we explore a special case in which the natural frequencies of the oscillators and the vertex degrees are linearly coupled. We find that in uncorrelated scale-free networks with the degree distribution exponent $2l\ensuremath{\gamma}l3$, the model undergoes a first-order phase transition, while the transition becomes second order at $\ensuremath{\gamma}g3$. If $\ensuremath{\gamma}=3$, the phase synchronization emerges as a result of a hybrid phase transition that combines an abrupt emergence of synchronization, as in first-order phase transitions, and a critical singularity, as in second-order phase transitions. The critical fluctuations manifest themselves as avalanches in the synchronization process. Comparing our analytical calculations with numerical simulations for Erd\ifmmode \mbox{\H{o}}\else \H{o}\fi{}s-R\'enyi and scale-free networks, we demonstrate that the annealed network approach is accurate if the mean degree and size of the network are sufficiently large. We also study analytically and numerically the Kuramoto model on star graphs and find that if the natural frequency of the central oscillator is sufficiently large in comparison to the average frequency of its neighbors, then synchronization emerges as a result of a first-order phase transition. This shows that oscillators sitting at hubs in a network may generate a discontinuous synchronization transition.
Spline regression for zero-inflated models

We propose a regression model for count data when the classical generalized linear model approach is too rigid due to a high outcome of zero counts and a nonlinear influence of continuous covariates. Zero-Inflation is applied to take into account the presence of excess zeros with separate link functions for the zero and the nonzero component. Nonlinearity in covariates is captured by spline functions based on B-splines. Our algorithm relies on maximum-likelihood estimation and allows for adaptive box-constrained knots, thus improving the goodness of the spline fit and allowing for detection of sensitivity changepoints. A simulation study substantiates the numerical stability of the algorithm to infer such models. The AIC criterion is shown to serve well for model selection, in particular if nonlinearities are weak such that BIC tends to overly simplistic models. We fit the introduced models to real data of children's dental sanity, linking caries counts with the so-called Body-Mass-Index (BMI) and other socioeconomic factors. This reveals a puzzling nonmonotonic influence of BMI on caries counts which is yet to be explained by clinical experts.
Fixed effects Selection in high dimensional Linear Mixed Models

We consider linear mixed models in which the observations are grouped. A L1-penalization on the fixed effects coefficients of the log-likelihood obtained by considering the random effects as missing values is proposed. A multicycle ECM algorithm is used to solve the optimization problem; it can be combined with any variable selection method developed for linear models. The algorithm allows the number of parameters p to be larger than the total number of observations n; it is faster than the lmmLasso (Schelldorfer,2011) since no n*n matrix has to be inverted. We show that the theoretical results of Schelldorfer (2011) apply for our method when the variances of both the random effects and the residuals are known. The combination of the algorithm with a variable selection method (Rohart 2011) shows good results in estimating the set of relevant fixed effects coefficients as well as estimating the variances; it outperforms the lmmLasso both in the common case (p n).
A particle filter approach to approximate posterior Cram\'er-Rao lower bound

The posterior Cram\'er-Rao lower bound (PCRLB) derived in Tichavsk\'y et al., 1998, provides a bound on the mean square error (MSE) obtained with any non-linear state filter. Computing the PCRLB involves solving complex, multi-dimensional expectations, which do not lend themselves to an easy analytical solution. Furthermore, any attempt to approximate it using numerical or simulation based approaches require a priori access to the true states, which may not be available, except in simulations or in carefully designed experiments. To allow recursive approximation of the PCRLB when the states are hidden or unmeasured, a new approach based on sequential Monte-Carlo (SMC) or particle filters (PF) is proposed. The approach uses SMC methods to estimate the hidden states using a sequence of the available sensor measurements. The developed method is general and can be used to approximate the PCRLB in non-linear systems with non-Gaussian state and sensor noise. The efficacy of the developed method is illustrated on two simulation examples, including a practical problem of ballistic target tracking at re-entry phase.
Bayesian estimation of possible causal direction in the presence of latent confounders using a linear non-Gaussian acyclic structural equation model with individual-specific effects

We consider learning the possible causal direction of two observed variables in the presence of latent confounding variables. Several existing methods have been shown to consistently estimate causal direction assuming linear or some type of nonlinear relationship and no latent confounders. However, the estimation results could be distorted if either assumption is actually violated. In this paper, we first propose a new linear non-Gaussian acyclic structural equation model with individual-specific effects that allows latent confounders to be considered. We then propose an empirical Bayesian approach for estimating possible causal direction using the new model. We demonstrate the effectiveness of our method using artificial and real-world data.
Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation

This article presents individual conditional expectation (ICE) plots, a tool for visualizing the model estimated by any supervised learning algorithm. Classical partial dependence plots (PDPs) help visualize the average partial relationship between the predicted response and one or more features. In the presence of substantial interaction effects, the partial response relationship can be heterogeneous. Thus, an average curve, such as the PDP, can obfuscate the complexity of the modeled relationship. Accordingly, ICE plots refine the PDP by graphing the functional relationship between the predicted response and the feature for individual observations. Specifically, ICE plots highlight the variation in the fitted values across the range of a covariate, suggesting where and to what extent heterogeneities might exist. In addition to providing a plotting suite for exploratory analysis, we include a visual test for additive structure in the data-generating model. Through simulated examples and real datasets, we demonstrate how ICE plots can shed light on estimated models in ways PDPs cannot. Procedures outlined are available in the R package ICEbox.
Recursive Pathways to Marginal Likelihood Estimation with Prior-Sensitivity Analysis

We investigate the utility to computational Bayesian analyses of a particular family of recursive marginal likelihood estimators characterized by the (equivalent) algorithms known as \biased sampling" or \reverse lo- gistic regression" in the statistics literature and \the density of states" in physics. Through a pair of numerical examples (including mixture modeling of the well-known galaxy dataset) we highlight the remarkable diversity of sampling schemes amenable to such recursive normalization, as well as the notable eciency of the resulting pseudo-mixture distributions for gauging prior-sensitivity in the Bayesian model selection context. Our key theo- retical contributions are to introduce a novel heuristic (\thermodynamic integration via importance sampling") for qualifying the role of the bridg- ing sequence in this procedure, and to reveal various connections between these recursive estimators and the nested sampling technique.