Miracle-dz/UniFAR-FTUs huggingface dataset:
Fast Embedding for JOFC Using the Raw Stress Criterion

ABSTRACT The joint optimization of fidelity and commensurability (JOFC) manifold matching methodology embeds an omnibus dissimilarity matrix consisting of multiple dissimilarities on the same set of objects. One approach to this embedding optimizes the preservation of fidelity to each individual dissimilarity matrix together with commensurability of each given observation across modalities via iterative majorization of a raw stress error criterion by successive Guttman transforms. In this article, we exploit the special structure inherent to JOFC to exactly and efficiently compute the successive Guttman transforms, and as a result we are able to greatly speed up the JOFC procedure for both in-sample and out-of-sample embedding. We demonstrate the scalability of our implementation on both real and simulated data examples.
Bootstrap uniform central limit theorems for Harris recurrent Markov chains

The main objective of this paper is to establish bootstrap uniform functional central limit theorem for Harris recurrent Markov chains over uniformly bounded classes of functions. We show that the result can be generalized also to the unbounded case. To avoid some complicated mixing conditions, we make use of the well-known regeneration properties of Markov chains. We show that in the atomic case the proof of the bootstrap uniform central limit theorem for Markov chains for functions dominated by a function in $L^{2}$ space proposed by Radulovi\'{c} (2004) can be significantly simplified. Finally, we prove bootstrap uniform central limit theorems for Fr\'{e}chet differentiable functionals in a Markovian setting.
Optimal rates for total variation denoising

Motivated by its practical success, we show that the two-dimensional total variation denoiser satisfies a sharp oracle inequality that leads to near optimal rates of estimation for a large class of image models such as bi-isotonic, H\"older smooth and cartoons. Our analysis hinges on properties of the unnormalized Laplacian of the two-dimensional grid such as eigenvector delocalization and spectral decay. We also present extensions to more than two dimensions as well as several other graphs.
Flexible online multivariate regression with variational Bayes and the matrix-variate Dirichlet process

Flexible regression methods where interest centres on the way that the whole distribution of a response vector changes with covariates are very useful in some applications. A recently developed technique in this regard uses the matrix-variate Dirichlet process as a prior for a mixing distribution on a coefficient in a multivariate linear regression model. The method is attractive, particularly in the multivariate setting, for the convenient way that it allows for borrowing strength across different component regressions and for its computational simplicity and tractability. The purpose of the present article is to develop fast online variational Bayes approaches to fitting this model and to investigate how they perform compared to MCMC and batch variational methods in a number of scenarios.
LOCO: Distributing Ridge Regression with Random Projections

We propose LOCO, an algorithm for large-scale ridge regression which distributes the features across workers on a cluster. Important dependencies between variables are preserved using structured random projections which are cheap to compute and must only be communicated once. We show that LOCO obtains a solution which is close to the exact ridge regression solution in the fixed design setting. We verify this experimentally in a simulation study as well as an application to climate prediction. Furthermore, we show that LOCO achieves significant speedups compared with a state-of-the-art distributed algorithm on a large-scale regression problem.
Improved quantification of naphthalene using non-linear Partial Least Squares Regression

A test dataset is generated using temperature cycled operation with a WO3 metal oxide semiconductor (MOS) gas sensor. Six concentrations of naphthalene from 0 to 40 ppb are measured and, subsequently, used to evaluate the performance of three variants of Partial Least Squares Regression (PLSR). Ordinary PLSR produces highly non-linear models due to the non-linear response of the sensor. Double-logarithmic data results in a model with much better linearity which has a resolution of 4 ppb in the range from 0 to 20 ppb. The more complex Locally Weighted PLSR (LW-PLSR) produces an even better model, especially for higher concentrations, without making any assumptions for relationships in the underlying data.
Remember the curse of dimensionality: the case of goodness-of-fit testing in arbitrary dimension

ABSTRACT Despite a substantial literature on nonparametric two-sample goodness-of-fit testing in arbitrary dimensions, there is no mention there of any curse of dimensionality. In fact, in some publications, a parametric rate is derived. As we discuss below, this is because a directional alternative is considered. Indeed, even in dimension one, Ingster, Y. I. [(1987). Minimax testing of nonparametric hypotheses on a distribution density in the l_p metrics. Theory of Probability & Its Applications, 31(2), 333–337] has shown that the minimax rate is not parametric. In this paper, we extend his results to arbitrary dimension and confirm that the minimax rate is not only nonparametric, exhibits but also a prototypical curse of dimensionality. We further extend Ingster's work to show that the chi-squared test achieves the minimax rate. Moreover, we show that the test adapts to the intrinsic dimensionality of the data. Finally, in the spirit of Ingster, Y. I. [(2000). Adaptive chi-square tests. Journal of Mathematical Sciences, 99(2), 1110–1119], we consider a multiscale version of the chi-square test, showing that one can adapt to unknown smoothness without much loss in power.
k-Means Clustering Is Matrix Factorization

We show that the objective function of conventional k-means clustering can be expressed as the Frobenius norm of the difference of a data matrix and a low rank approximation of that data matrix. In short, we show that k-means clustering is a matrix factorization problem. These notes are meant as a reference and intended to provide a guided tour towards a result that is often mentioned but seldom made explicit in the literature.
Indian Buffet process for model selection in convolved multiple-output Gaussian processes

Multi-output Gaussian processes have received increasing attention during the last few years as a natural mechanism to extend the powerful flexibility of Gaussian processes to the setup of multiple output variables. The key point here is the ability to design kernel functions that allow exploiting the correlations between the outputs while fulfilling the positive definiteness requisite for the covariance function. Alternatives to construct these covariance functions are the linear model of coregionalization and process convolutions. Each of these methods demand the specification of the number of latent Gaussian process used to build the covariance function for the outputs. We propose in this paper, the use of an Indian Buffet process as a way to perform model selection over the number of latent Gaussian processes. This type of model is particularly important in the context of latent force models, where the latent forces are associated to physical quantities like protein profiles or latent forces in mechanical systems. We use variational inference to estimate posterior distributions over the variables involved, and show examples of the model performance over artificial data, a motion capture dataset, and a gene expression dataset.
The Midpoint Mixed Model with a Missingness Mechanism (M5): A Likelihood-Based Framework for Quantification of Mass Spectrometry Proteomics Data (Preprint)

Statistical models for proteomics data often estimate protein fold changes between two samples, A and B, as the average peptide intensity from sample A divided by the average peptide intensity from sample B. Such average intensity ratios fail to take full advantage of the experimental design which eliminates unseen confounding variables by processing peptides from both samples under identical conditions. Typically this structure is exploited through the estimation of a protein ratio as the median ratio of matched peptide intensities. This simple solution fails to account for a substantial missing data bias which has led to the development of more sophisticated average intensity models. Here we develop the first statistical model that accounts for nonignorable missingness while utilizing peptide level matched pairs across samples. Our simulation analysis shows that models which fail to utilize peptide level ratios, su.er astonishing losses to accuracy with basic ANOVA estimates having an average MSE 371% higher than median ratio estimates. In turn, median ratio estimates have an average MSE 35% higher than our model estimates. An analysis of breast cancer data reinforces these relationships and shows that our model is capable of increasing the number of proteins estimated by 22%.
Stepwise Choice of Covariates in High Dimensional Regression

Given data y(n) and p(n)covariates x(n) one problem in linear regression is to decide which if any of the covariates to include. There are many articles on this problem but all are based on a stochastic model for the data. This paper gives what seems to be a new approach which does not require any form of model. It is conceptually and algorithmically simple and consistency results can be proved under appropriate assumptions.
Variational Inference with Hamiltonian Monte Carlo

Variational inference lies at the core of many state-of-the-art algorithms. To improve the approximation of the posterior beyond parametric families, it was proposed to include MCMC steps into the variational lower bound. In this work we explore this idea using steps of the Hamiltonian Monte Carlo (HMC) algorithm, an efficient MCMC method. In particular, we incorporate the acceptance step of the HMC algorithm, guaranteeing asymptotic convergence to the true posterior. Additionally, we introduce some extensions to the HMC algorithm geared towards faster convergence. The theoretical advantages of these modifications are reflected by performance improvements in our experimental results.
Robust Kernel (Cross-) Covariance Operators in Reproducing Kernel Hilbert Space toward Kernel Methods

To the best of our knowledge, there are no general well-founded robust methods for statistical unsupervised learning. Most of the unsupervised methods explicitly or implicitly depend on the kernel covariance operator (kernel CO) or kernel cross-covariance operator (kernel CCO). They are sensitive to contaminated data, even when using bounded positive definite kernels. First, we propose robust kernel covariance operator (robust kernel CO) and robust kernel crosscovariance operator (robust kernel CCO) based on a generalized loss function instead of the quadratic loss function. Second, we propose influence function of classical kernel canonical correlation analysis (classical kernel CCA). Third, using this influence function, we propose a visualization method to detect influential observations from two sets of data. Finally, we propose a method based on robust kernel CO and robust kernel CCO, called robust kernel CCA, which is designed for contaminated data and less sensitive to noise than classical kernel CCA. The principles we describe also apply to many kernel methods which must deal with the issue of kernel CO or kernel CCO. Experiments on synthesized and imaging genetics analysis demonstrate that the proposed visualization and robust kernel CCA can be applied effectively to both ideal data and contaminated data. The robust methods show the superior performance over the state-of-the-art methods.
Variational analysis of inference from dynamical systems

We introduce and study a variational framework for the analysis of empirical risk based inference for dynamical systems and ergodic processes. The analysis applies to a two-stage estimation procedure in which (i) the trajectory of an observed (but unknown) system is fit to a trajectory from a known reference system by minimizing cumulative per-state loss, and (ii) a parameter estimate is obtained from the initial state of the best fit reference trajectory. We show that the empirical risk of the best fit trajectory converges almost surely to a constant that can be expressed in a variational form as the minimal expected loss over dynamically invariant couplings (joinings) of the observed and reference systems. Moreover, we establish that the family of joinings minimizing the expected loss is convex and compact, and that it fully characterizes the asymptotic behavior of the estimated parameters, addressing both identifiability and misspecification. The two-stage estimation framework and associated variational analysis apply to a broad family of empirical risk miminization procedures for dependent observations. To illustrate this, we apply variational analysis to the well studied problems of maximum likelihood and non-linear regression, and then undertake an extended analysis of system identification from quantized trajectories subject to noise, a problem of interest in dynamics, where the models themselves exhibit dynamical behavior across time.
Analysis of an indirect (mediated) moderation model

The main purpose of this research is to investigate an integration of mediation and moderation. When the effect of X on Y is moderated by Z, does this moderating effect still exist when we include another moderator W, which is related to Z. In contrast to a mediated effect being moderated in current integrated models, here the moderating effect is mediated. We simulated and analyzed data using R under several conditions and checked the times the moderation by Z was actually mediated by W. We set up a four step decision-tree which guides the user trough the steps of the regression analyses in order to infer or refute mediated moderation.
Application of Bootstrap Re-sampling Method to a Categorical Data of HIV/AIDS Spread across different Social-Economic Classes

This research reports on the relationship and significance of social-economic factors (age, sex, employment status) and modes of HIV/AIDS transmission to the HIV/AIDS spread. Logistic regression model, a form of probabilistic function for binary response was used to relate social-economic factors (age, sex, employment status) to HIV/AIDS spread. The statistical predictive model was used to project the likelihood response of HIV/AIDS spread with a larger population using 10,000 Bootstrap re-sampling observations.
Best linear unbiased estimators in continuous time regression models

In this paper the problem of best linear unbiased estimation is investigated for continuous-time regression models. We prove several general statements concerning the explicit form of the best linear unbiased estimator (BLUE), in particular when the error process is a smooth process with one or several derivatives of the response process available for construction of the estimators. We derive the explicit form of the BLUE for many specific models including the cases of continuous autoregressive errors of order two and integrated error processes (such as integrated Brownian motion). The results are illustrated by several examples.
Adaptive scaling for soft-thresholding estimator

Soft-thresholding is a sparse modeling method that is typically applied to wavelet denoising in statistical signal processing and analysis. It has a single parameter that controls a threshold level on wavelet coefficients and, simultaneously, amount of shrinkage for coefficients of un-removed components. This parametrization is possible to cause excess shrinkage, thus, estimation bias at a sparse representation; i.e. there is a dilemma between sparsity and prediction accuracy. To relax this problem, we considered to introduce positive scaling on soft-thresholding estimator, by which threshold level and amount of shrinkage are independently controlled. Especially, in this paper, we proposed component-wise and data-dependent scaling in a setting of non-parametric orthogonal regression problem including discrete wavelet transform. We call our scaling method adaptive scaling. We here employed soft-thresholding method based on LARS(least angle regression), by which the model selection problem reduces to the determination of the number of un-removed components. We derived a risk under LARS-based soft-thresholding with the proposed adaptive scaling and established a model selection criterion as an unbiased estimate of the risk. We also analyzed some properties of the risk curve and found that the model selection criterion is possible to select a model with low risk and high sparsity compared to a naive soft-thresholding method. This theoretical speculation was verified by a simple numerical experiment and an application to wavelet denoising.
On metrizing vague convergence of random measures with applications on Bayesian nonparametric models

ABSTRACT This paper deals with studying vague convergence of random measures of the form , where is a sequence of independent and identically distributed random variables with common distribution Π, denotes the Dirac measure at and are random variables, independent of , chosen according to certain procedures such that almost surely, as , for fixed i. We show that, as , converges vaguely almost surely to if and only if converges vaguely almost surely to for all k fixed. The limiting process μ plays a central role in many areas in statistics, including Bayesian nonparametric models. A finite approximation of the beta process is derived from the application of this result. A simulated example is incorporated, in which the proposed approach exhibits an excellent performance over several existing algorithms.
A Coherent Integration Method Based on Radon Non-uniform FRFT for Random Pulse Repetition Interval (RPRI) Radar

To solve the range cell migration (RCM) and spectrum spread during the integration time induced by the motion of a target, this paper proposes a new coherent integration method based on Radon non-uniform FRFT (NUFRFT) for random pulse repetition interval (RPRI) radar. In this method, RCM is eliminated via searching in the motion parameters space and the spectrum spread is resolved by using NUFRFT. Comparisons with other popular methods, moving target detection (MTD), Radon-Fourier transform (RFT), and Radon-Fractional Fourier Transform (RFRFT) are performed. The simulation results demonstrate that the proposed method can detect the moving target even in low SNR scenario and is superior to the other two methods.
A Novel Sparsity-Based Approach to Recursive Estimation of Dynamic Parameter Sets

We consider the problem of estimating a variable number of parameters with a dynamic nature. A familiar example is finding the position of moving targets using sensor array observations. The problem is challenging in cases where either the observations are not reliable or the parameters evolve rapidly. Inspired by the sparsity based techniques, we introduce a novel Bayesian model for the problems of interest and study its associated recursive Bayesian filter. We propose an algorithm approximating the Bayesian filter, maintaining a reasonable amount of calculations. We compare by numerical evaluation the resulting technique to state-of-the-art algorithms in different scenarios. In a scenario with a low SNR, the proposed method outperforms other complex techniques.
Efficiency of Z-estimators indexed by the objective functions

We study the convergence of $Z$-estimators $\widehat \theta(\eta)\in \mathbb R^p$ for which the objective function depends on a parameter $\eta$ that belongs to a Banach space $\mathcal H$. Our results include the uniform consistency over $\mathcal H$ and the weak convergence in the space of bounded $\mathbb R^p$-valued functions defined on $\mathcal H$. Furthermore when $\eta$ is a tuning parameter optimally selected at $\eta_0$, we provide conditions under which an estimated $\widehat \eta$ can be replaced by $\eta_0$ without affecting the asymptotic variance. Interestingly, these conditions are free from any rate of convergence of $\widehat \eta$ to $\eta_0$ but they require the space described by $\widehat \eta$ to be not too large. We highlight several applications of our results and we study in detail the case where $\eta$ is the weight function in weighted regression.
Submodularity in Statistics: Comparing the Success of Model Selection Methods

We demonstrate the usefulness of submodularity in statistics as a characterization of the difficulty of the search problem of feature selection. The search problem is the ability of a procedure to identify an informative set of features as opposed to the performance of the optimal set of features. Submodularity arises naturally in this setting due to its connection to combinatorial optimization. In statistics, submodularity isolates cases in which collinearity makes the choice of model features difficult from those in which this task is routine. Researchers often report the signal-to-noise ratio to measure the difficulty of simulated data examples. A measure of submodularity should also be provided as it characterizes an independent component difficulty. Furthermore, it is closely related to other statistical assumptions used in the development of the Lasso, Dantzig selector, and sure information screening.
Modification of the MDR-EFE method for stratified samples

The MDR-EFE method of performing identification of relevant factors within a given collection X_1,. ,X_n is developed for stratified samples in the case of binary response variable Y. We establish a criterion of strong consistency of estimates (involving K-cross-validation procedure and penalty) for a specified prediction error function. The cost approach is proposed to compare experiments with random and nonrandom number of observations. Analytic results and simulations demonstrate advantages of the method introduced for stratified samples over that employed for i.i. d. learning sample.
Limitations of Design-based Causal Inference and A/B Testing under Arbitrary and Network Interference

Randomized experiments on a network often involve interference between connected units, namely, a situation in which an individual’s treatment can affect the response of another individual. Current approaches to deal with interference, in theory and in practice, often make restrictive assumptions on its structure—for instance, assuming that interference is local—even when using otherwise nonparametric inference strategies. This reliance on explicit restrictions on the interference mechanism suggests a shared intuition that inference is impossible without any assumptions on the interference structure. In this paper, we begin by formalizing this intuition in the context of a classical nonparametric approach to inference, referred to as design-based inference of causal effects. Next, we show how, always in the context of design-based inference, even parametric structural assumptions that allow the existence of unbiased estimators cannot guarantee a decreasing variance even in the large sample limit. This lack of concentration in large samples is often observed empirically, in randomized experiments in which interference of some form is expected to be present. This result has direct consequences for the design and analysis of large experiments—for instance, in online social platforms—where the belief is that large sample sizes automatically guarantee small variance. More broadly, our results suggest that although strategies for causal inference in the presence of interference borrow their formalism and main concepts from the traditional causal inference literature, much of the intuition from the no-interference case do not easily transfer to the interference setting.