Miracle-dz/UniFAR-FTUs huggingface dataset:
A stochastic process approach to false discovery control

This paper extends the theory of false discovery rates (FDR) pioneered by Benjamini and Hochberg [J. Roy. Statist. Soc. Ser B 57 (1995) 289-300]. We develop a framework in which the False Discovery Proportion (FDP)-the number of false rejections divided by the number of rejections-is treated as a stochastic process. After obtaining the limiting distribution of the process, we demonstrate the validity of a class of procedures for controlling the False Discovery Rate (the expected FDP). We construct a confidence envelope for the whole FDP process. From these envelopes we derive confidence thresholds, for controlling the quantiles of the distribution of the FDP as well as controlling the number of false discoveries. We also investigate methods for estimating the p-value distribution.
Prediction regions through Inverse Regression

Predict a new response from a covariate is a challenging task in regression, which raises new question since the era of high-dimensional data. In this paper, we are interested in the inverse regression method from a theoretical viewpoint. Theoretical results have already been derived for the well-known linear model, but recently, the curse of dimensionality has increased the interest of practitioners and theoreticians into generalization of those results for various estimators, calibrated for the high-dimension context. To deal with high-dimensional data, inverse regression is used in this paper. It is known to be a reliable and efficient approach when the number of features exceeds the number of observations. Indeed, under some conditions, dealing with the inverse regression problem associated to a forward regression problem drastically reduces the number of parameters to estimate and make the problem tractable. When both the responses and the covariates are multivariate, estimators constructed by the inverse regression are studied in this paper, the main result being explicit asymptotic prediction regions for the response. The performances of the proposed estimators and prediction regions are also analyzed through a simulation study and compared with usual estimators.
Estimating Differential Entropy under Gaussian Convolutions

This paper studies the problem of estimating the differential entropy $h(S+Z)$, where $S$ and $Z$ are independent $d$-dimensional random variables with $Z\sim\mathcal{N}(0,\sigma^2 \mathrm{I}_d)$. The distribution of $S$ is unknown, but $n$ independently and identically distributed (i.i.d) samples from it are available. The question is whether having access to samples of $S$ as opposed to samples of $S+Z$ can improve estimation performance. We show that the answer is positive. More concretely, we first show that despite the regularizing effect of noise, the number of required samples still needs to scale exponentially in $d$. This result is proven via a random-coding argument that reduces the question to estimating the Shannon entropy on a $2^{O(d)}$-sized alphabet. Next, for a fixed $d$ and $n$ large enough, it is shown that a simple plugin estimator, given by the differential entropy of the empirical distribution from $S$ convolved with the Gaussian density, achieves the loss of $O\left((\log n)^{d/4}/\sqrt{n}\right)$. Note that the plugin estimator amounts here to the differential entropy of a $d$-dimensional Gaussian mixture, for which we propose an efficient Monte Carlo computation algorithm. At the same time, estimating $h(S+Z)$ via popular differential entropy estimators (based on kernel density estimation (KDE) or k nearest neighbors (kNN) techniques) applied to samples from $S+Z$ would only attain much slower rates of order $O(n^{-1/d})$, despite the smoothness of $P_{S+Z}$. As an application, which was in fact our original motivation for the problem, we estimate information flows in deep neural networks and discuss Tishby's Information Bottleneck and the compression conjecture, among others.
Selection of proposal distributions for generalized importance sampling estimators

The standard importance sampling (IS) method uses samples from a single proposal distribution and assigns weights to them, according to the ratio of the target and proposal pdfs. This naive IS estimator, generally does not work well in multiple targets examples as the weights can take arbitrarily large values making the estimator highly unstable. In such situations, alternative generalized IS estimators involving samples from multiple proposal distributions are preferred. Just like the standard IS, the success of these multiple IS estimators crucially depends on the choice of the proposal distributions. The selection of these proposal distributions is the focus of this article. We propose three methods based on (i)~a geometric space filling coverage criterion, (ii)~a sequential maximum variance approach, and (iii)~a maximum entropy approach. In particular, we describe these methods in the context of Doss's (2010) two-stage IS estimator, although the first two methods are applicable to any multi-proposal IS estimator. Two of the proposed approaches use estimates of asymptotic variances of Geyer's (1994) reverse logistic estimator and Doss's (2010) IS estimators. Thus, we provide consistent spectral variance estimators for these asymptotic variances. We demonstrate the performance of these spectral variance methods for estimating the size parameter of a negative binomial generalized linear model. The proposed methods for selecting proposal densities are illustrated using two detailed examples. The first example is a robust Bayesian binary regression model, where the generalized IS method is used to estimate the link function parameter. The second example involves analysis of count data using the Poisson spatial generalized linear mixed model.
Practical Heteroscedastic Gaussian Process Modeling for Large Simulation Experiments

ABSTRACT We present a unified view of likelihood based Gaussian progress regression for simulation experiments exhibiting input-dependent noise. Replication plays an important role in that context, however previous methods leveraging replicates have either ignored the computational savings that come from such design, or have short-cut full likelihood-based inference to remain tractable. Starting with homoscedastic processes, we show how multiple applications of a well-known Woodbury identity facilitate inference for all parameters under the likelihood (without approximation), bypassing the typical full-data sized calculations. We then borrow a latent-variable idea from machine learning to address heteroscedasticity, adapting it to work within the same thrifty inferential framework, thereby simultaneously leveraging the computational and statistical efficiency of designs with replication. The result is an inferential scheme that can be characterized as single objective function, complete with closed form derivatives, for rapid library-based optimization. Illustrations are provided, including real-world simulation experiments from manufacturing and the management of epidemics.
Reparametrization of COM–Poisson regression models with applications in the analysis of experimental data

The COM–Poisson distribution is a two-parameter generalization of the Poisson distribution that can deal with under-, equi- and overdispersed count data. Unfortunately, its location parameter does not correspond to the expectation, which complicates the parameter interpretation. In this article, we propose a straightforward reparametrization of the COM–Poisson distribution based on an approximation to the expectation. Estimation and inference are done using the likelihood paradigm. Simulation studies show that the maximum likelihood estimators are unbiased and consistent for both regression and dispersion parameters. In addition, the nature of the deviance surfaces suggests that these parameters are also orthogonal for most of the parameter space, which is advantageous for interpretation, inference and computational efficiency. Study of the distribution’s properties, through a consideration of dispersion, zero-inflation and heavy tail indexes, together with the results of data analyses show the flexibility over standard approaches. The computational routines and datasets are available in the supplementary material.
Development of ICA and IVA Algorithms with Application to Medical Image Analysis

Independent component analysis (ICA) is a widely used BSS method that can uniquely achieve source recovery, subject to only scaling and permutation ambiguities, through the assumption of statistical independence on the part of the latent sources. Independent vector analysis (IVA) extends the applicability of ICA by jointly decomposing multiple datasets through the exploitation of the dependencies across datasets. Though both ICA and IVA algorithms cast in the maximum likelihood (ML) framework enable the use of all available statistical information in reality, they often deviate from their theoretical optimality properties due to improper estimation of the probability density function (PDF). This motivates the development of flexible ICA and IVA algorithms that closely adhere to the underlying statistical description of the data. Although it is attractive minimize the assumptions, important prior information about the data, such as sparsity, is usually available. If incorporated into the ICA model, use of this additional information can relax the independence assumption, resulting in an improvement in the overall separation performance. Therefore, the development of a unified mathematical framework that can take into account both statistical independence and sparsity is of great interest. In this work, we first introduce a flexible ICA algorithm that uses an effective PDF estimator to accurately capture the underlying statistical properties of the data. We then discuss several techniques to accurately estimate the parameters of the multivariate generalized Gaussian distribution, and how to integrate them into the IVA model. Finally, we provide a mathematical framework that enables direct control over the influence of statistical independence and sparsity, and use this framework to develop an effective ICA algorithm that can jointly exploit these two forms of diversity.
Sufficient Conditions for a Linear Estimator to be a Local Polynomial Regression

It is shown that any linear estimator that satisfies the moment conditions up to order $p$ is equivalent to a local polynomial regression of order $p$ with some non-negative weight function if and only if the kernel has at most $p$ sign changes. If the data points are placed symmetrically about the estimation point, a linear weighting function is equivalent to the standard quadratic weighting function.
Modeling goal chances in soccer: a Bayesian inference approach

We consider the task of determining the number of chances a soccer team creates, along with the composite nature of each chance-the players involved and the locations on the pitch of the assist and the chance. We propose an interpretable Bayesian inference approach and implement a Poisson model to capture chance occurrences, from which we infer team abilities. We then use a Gaussian mixture model to capture the areas on the pitch a player makes an assist/takes a chance. This approach allows the visualization of differences between players in the way they approach attacking play (making assists/taking chances). We apply the resulting scheme to the 2016/2017 English Premier League, capturing team abilities to create chances, before highlighting key areas where players have most impact.
Canonical Least Favorable Submodels:A New TMLE Procedure for Multidimensional Parameters

This paper is a fundamental addition to the world of targeted maximum likelihood estimation (TMLE) (or likewise, targeted minimum loss estimation) for simultaneous estimation of multi-dimensional parameters of interest. TMLE, as part of the targeted learning framework, offers a crucial step in constructing efficient plug-in estimators for nonparametric or semiparametric models. The so-called targeting step of targeted learning, involves fluctuating the initial fit of the model in a way that maximally adjusts the plug-in estimate per change in the log likelihood. Previously for multidimensional parameters of interest, iterative TMLE's were constructed using locally least favorable submodels as defined in van der Laan and Gruber, 2016, which are indexed by a multidimensional fluctuation parameter. In this paper we define a canonical least favorable submodel in terms of a single dimensional epsilon for a $d$-dimensional parameter of interest. One can view the clfm as the iterative analog to the one-step TMLE as constructed in van der Laan and Gruber, 2016. It is currently implemented in several software packages we provide in the last section. Using a single epsilon for the targeting step in TMLE could be useful for high dimensional parameters, where using a fluctuation parameter of the same dimension as the parameter of interest could suffer the consequences of curse of dimensionality. The clfm also enables placing the so-called clever covariate denominator as an inverse weight in an offset intercept model. It has been shown that such weighting mitigates the effect of large inverse weights sometimes caused by near positivity violations.
Joint training framework for text-to-speech and voice conversion using multi-source Tacotron and WaveNet

We investigated the training of a shared model for both text-to-speech (TTS) and voice conversion (VC) tasks. We propose using an extended model architecture of Tacotron, that is a multi-source sequence-to-sequence model with a dual attention mechanism as the shared model for both the TTS and VC tasks. This model can accomplish these two different tasks respectively according to the type of input. An end-to-end speech synthesis task is conducted when the model is given text as the input while a sequence-to-sequence voice conversion task is conducted when it is given the speech of a source speaker as the input. Waveform signals are generated by using WaveNet, which is conditioned by using a predicted mel-spectrogram. We propose jointly training a shared model as a decoder for a target speaker that supports multiple sources. Listening experiments show that our proposed multi-source encoder-decoder model can efficiently achieve both the TTS and VC tasks.
Some variations on Ensembled Random Survival Forest with application to Cancer Research

In this paper we describe a novel implementation of adaboost for prediction of survival function. We take different variations of the algorithm and compare the algorithms based on system run time and root mean square error. Our construction includes right censoring data and competing risk data too. We take different data set to illustrate the performance of the algorithms.
Model Criticism for Bayesian Causal Inference

The goal of causal inference is to understand the outcome of alternative courses of action. However, all causal inference requires assumptions. Such assumptions can be more influential than in typical tasks for probabilistic modeling, and testing those assumptions is important to assess the validity of causal inference. We develop model criticism for Bayesian causal inference, building on the idea of posterior predictive checks to assess model fit. Our approach involves decomposing the problem, separately criticizing the model of treatment assignments and the model of outcomes. Conditioned on the assumption of unconfoundedness---that the treatments are assigned independently of the potential outcomes---we show how to check any additional modeling assumption. Our approach provides a foundation for diagnosing model-based causal inferences.
Optimal Temperature Spacing for Regionally Weight-preserving Tempering

Parallel tempering is popular method for allowing MCMC algorithms to properly explore a $d$-dimensional multimodal target density. One problem with traditional power-based parallel tempering for multimodal targets is that the proportion of probability mass associated to the modes can change for different inverse-temperature values, sometimes dramatically so. Complementary work by the authors proposes a novel solution involving auxiliary targets that preserve regional weight upon powering up the density. This paper attempts to address the question of how to choose the temperature spacings in an optimal way when using this type of weight-preserving approach.The problem is analysed in a tractable setting for computation of the expected squared jumping distance which can then be optimised with regards to a tuning parameter. The conclusion is that for an appropriately constructed regionally weight-preserved tempering algorithm targeting a $d$-dimensional target distribution, the consecutive temperature spacings should behave as $\mathcal{O}\left(d^{-1/2}\right)$ and this induces an optimal acceptance rate for temperature swap moves that lies in the interval $[0,0.234]$.
Ball: An R Package for Detecting Distribution Difference and Association in Metric Spaces

The rapid development of modern technology facilitates the appearance of numerous unprecedented complex data which do not satisfy the axioms of Euclidean geometry, while most of the statistical hypothesis tests are available in Euclidean or Hilbert spaces. To properly analyze the data of more complicated structures, efforts have been made to solve the fundamental test problems in more general spaces. In this paper, a publicly available R package Ball is provided to implement Ball statistical test procedures for K-sample distribution comparison and test of mutual independence in metric spaces, which extend the test procedures for two sample distribution comparison and test of independence. The tailormade algorithms as well as engineering techniques are employed on the Ball package to speed up computation to the best of our ability. Two real data analyses and several numerical studies have been performed and the results certify the powerfulness of Ball package in analyzing complex data, e.g., spherical data and symmetric positive matrix data.
Sparse Multivariate ARCH Models: Finite Sample Properties

We provide finite sample properties of sparse multivariate ARCH processes, where the linear representation of ARCH models allows for an ordinary least squares estimation. Under the restricted strong convexity of the unpenalized loss function, regularity conditions on the penalty function, strict stationary and beta-mixing process, we prove non-asymptotic error bounds on the regularized ARCH estimator. Based on the primal-dual witness method of Loh and Wainwright (2017), we establish variable selection consistency, including the case when the penalty function is non-convex. These theoretical results are supported by empirical studies.
Acoustically Grounded Word Embeddings for Improved Acoustics-to-word Speech Recognition

Direct acoustics-to-word (A2W) systems for end-to-end automatic speech recognition are simpler to train, and more efficient to decode with, than sub-word systems. However, A2W systems can have difficulties at training time when data is limited, and at decoding time when recognizing words outside the training vocabulary. To address these shortcomings, we investigate the use of recently proposed acoustic and acoustically grounded word embedding techniques in A2W systems. The idea is based on treating the final pre-softmax weight matrix of an AWE recognizer as a matrix of word embedding vectors, and using an externally trained set of word embeddings to improve the quality of this matrix. In particular we introduce two ideas: (1) Enforcing similarity at training time between the external embeddings and the recognizer weights, and (2) using the word embeddings at test time for predicting out-of-vocabulary words. Our word embedding model is acoustically grounded, that is it is learned jointly with acoustic embeddings so as to encode the words’ acoustic-phonetic content; and it is parametric, so that it can embed any arbitrary (potentially out-of-vocabulary) sequence of characters. We find that both techniques improve the performance of an A2W recognizer on conversational telephone speech.
The Focused Information Criterion for Stochastic Model Selection Problems Using $M$-Estimators

Claeskens and Hjort (2003) constructed the focused information criterion (FIC) and developed frequentist model averaging methods using maximum likelihood estimators assuming the observations to be independent and identically distributed. Towards the immediate extensions and generalizations of these results, the present article is aimed at providing the focused model selection and model averaging methods using general maximum likelihood type estimators, popularly known as $M$-estimators. The necessary asymptotic theory is derived in a setup of stationary and strong mixing stochastic processes employing von Mises functional calculus of empirical processes and Le Cam's contiguity lemmas. We illustrate the proposed focused stochastic modeling methods using three well-known spacial cases of $M$-estimators, namely, conditional maximum likelihood estimators, conditional least square estimators and estimators based on method of moments. For the sake of simulation exercises, we consider two simple applications of FIC. The first application discusses the simultaneous selection of order of autoregression and symmetry of innovations in asymmetric Laplace autoregressive models. The second application demonstrates the FIC based choice between general scale-shape Gamma density and exponential density with shape being unity. We observe that in terms of the correct selections, FIC outperforms classical Akaike's information criterion AIC and performs at par with Bayesian information criterion BIC.
Multiply robust two-sample instrumental variable estimation

Although instrumental variable (IV) methods are widely used to estimate causal effects in the presence of unmeasured confounding, the IVs, exposure and outcome are often not measured in the same sample due to complex data harvesting procedures. Following the influential articles by Klevmarken (1982) and Angrist & Krueger (1992, 1995), numerous empirical researchers have applied two-sample IV methods to perform joint estimation based on an IV-exposure sample and an IV-outcome sample. This paper develops a general semi-parametric framework for two-sample data combination models from a missing data perspective, and characterizes the efficiency bound based on the full data model. In the context of the two-sample IV problem as a specific example, the framework offers insights on issues of efficiency and robustness of existing estimators. We propose new multiply robust locally efficient estimators of the causal effect of exposure on the outcome, and illustrate the methods through simulation and an econometric application on public housing projects.
On an Extension of Stochastic Approximation EM Algorithm for Incomplete Data Problems

The Stochastic Approximation EM (SAEM) algorithm, a variant stochastic approximation of EM, is a versatile tool for inference in incomplete data models. In this paper, we review the fundamental EM algorithm and then focus especially on the stochastic version of EM. In order to construct the SAEM, the algorithm combines EM with a variant of stochastic approximation that uses Markov chain Monte-Carlo to deal with the missing data. The algorithm is introduced in general form and can be used to a wide range of problems.