Miracle-dz/UniFAR-FTUs huggingface dataset:
Fast moment‐based estimation for hierarchical models

Hierarchical models allow for heterogeneous behaviours in a population while simultaneously borrowing estimation strength across all subpopulations. Unfortunately, existing likelihood‐based methods for fitting hierarchical models have high computational demands, and these demands have limited their adoption in large‐scale prediction and inference problems. The paper proposes a moment‐based procedure for estimating the parameters of a hierarchical model which has its roots in a method originally introduced by Cochran in 1937. The method trades statistical efficiency for computational efficiency. It gives consistent parameter estimates, competitive prediction error performance and substantial computational improvements. When applied to a large‐scale recommender system application and compared with a standard maximum likelihood procedure, the method delivers competitive prediction performance while reducing the sequential computation time from hours to minutes.
SHARP BOUNDS ON THE VARIANCE IN RANDOMIZED EXPERIMENTS

We propose a consistent estimator of sharp bounds on the variance of the difference-in-means estimator in completely randomized experiments. Generalizing Robins [Stat. Med. 7 (1988) 773-785], our results resolve a well-known identification problem in causal inference posed by Neyman [Statist. Sci. 5 (1990) 465-472. Reprint of the original 1923 paper]. A practical implication of our results is that the upper bound estimator facilitates the asymptotically narrowest conservative Wald-type confidence intervals, with applications in randomized controlled and clinical trials.
Explicit Evaluations of Matrix-variate Gamma and Beta Integrals in the Real and Complex Cases

Matrix transformations in terms of triangular matrices is the easiest method of evaluating matrix-variate gamma and beta integrals in the real and complex cases. Here we give several procedures of explicit evaluation of gamma and beta integrals in the general real and complex situations. The procedure also reveals the structure of these matrix-variate integrals. Apart from the evaluation of matrix-variate gamma and beta integrals, the procedure can also be applied to evaluate such integrals explicitly in similar situations. Various methods described here will be useful to those who are working on integrals involving real-valued scalar functions of matrix argument in general and gamma and beta integrals in particular.
Goodness-of-fit tests based on sup-functionals of weighted empirical processes

A large class of goodness-of-fit test statistics based on sup-functionals of weighted empirical processes is proposed and studied. The weight functions employed are Erd\H{o}s-Feller-Kolmogorov-Petrovski upper-class functions of a Brownian bridge. Based on the result of M. Cs\"{o}rg\H{o}, S. Cs\"{o}rg\H{o}, Horv\'{a}th, and Mason obtained for this type of test statistics, we provide the asymptotic null distribution theory for the class of tests in hand, and present an algorithm for tabulating the limit distribution functions under the null hypothesis. A new family of nonparametric confidence bands is constructed for the true distribution function and it is found to perform very well. The results obtained, together with a new result on the convergence in distribution of the higher criticism statistic, introduced by Donoho and Jin, demonstrate the advantage of our approach over a common approach that utilizes a family of regularly varying weight functions. Furthermore, we show that, in various subtle problems of detecting sparse heterogeneous mixtures, the proposed test statistics achieve the detection boundary found by Ingster and, when distinguishing between the null and alternative hypotheses, perform optimally adaptively to unknown sparsity and size of the non-null effects.
A Bayesian estimation approach to analyze non-Gaussian data-generating processes with latent classes

A large amount of observational data has been accumulated in various fields in recent times, and there is a growing need to estimate the generating processes of these data. A linear non-Gaussian acyclic model (LiNGAM) based on the non-Gaussianity of external influences has been proposed to estimate the data-generating processes of variables. However, the results of the estimation can be biased if there are latent classes. In this paper, we first review LiNGAM, its extended model, as well as the estimation procedure for LiNGAM in a Bayesian framework. We then propose a new Bayesian estimation procedure that solves the problem.
A New Class of Nonsymmetric Multivariate Dependence Measures

Following our previous work on copula-based nonsymmetric bivariate dependence measures, we propose a new set of conditions on nonsymmetric multivariate dependence measures which characterize both independence and complete dependence of one random variable on a group of random variables. The measures are nonparametric in that they are copula-based and are invariant under continuous bijective transformations on the group of random variables. We also construct explicitly new measures that satisfy the conditions. Besides, we extend the star product on bivariate copulas to multivariate copulas and prove the DPI condition and self-equitability for the new measures. A further extension to measures of dependence of one group of random variables on another group of random variables is also discussed.
Lecture notes on ridge regression

The linear regression model cannot be fitted to high-dimensional data, as the high-dimensionality brings about empirical non-identifiability. Penalized regression overcomes this non-identifiability by augmentation of the loss function by a penalty (i.e. a function of regression coefficients). The ridge penalty is the sum of squared regression coefficients, giving rise to ridge regression. Here many aspect of ridge regression are reviewed e.g. moments, mean squared error, its equivalence to constrained estimation, and its relation to Bayesian regression. Finally, its behaviour and use are illustrated in simulation and on omics data. Subsequently, ridge regression is generalized to allow for a more general penalty. Finally, the framework is translated to logistic regression and its properties are shown to carry over.
Transdimensional transformation based Markov chain Monte Carlo

In this article, we propose a novel and general dimension-hopping MCMC methodology that can update all the parameters as well as the number of parameters simultaneously using simple deterministic transformations of some low-dimensional (often one-dimensional) random variable. This methodology, which has been inspired by the recent Transformation based MCMC (TMCMC) for updating all the parameters simultaneously in general fixed-dimensional set-ups using low-dimensional random variables, facilitates great speed in terms of computation time and provides high acceptance rates, thanks to the low-dimensional random variables which effectively reduce the dimension dramatically. Quite importantly, our transformation based approach provides a natural way to automate the move-types in the variable dimensional problems. We refer to this methodology as Transdimensional Transformation based Markov Chain Monte Carlo (TTMCMC). We develop the theory of TTMCMC, illustrating it with gamma and normal mixtures with unknown number of components, for both simulated and real data sets. Comparisons with RJMCMC demonstrates far superior performance of TTMCMC in terms of mixing, acceptance rate, computational speed and automation. Furthermore, we demonstrate good performance of TTMCMC in multivariate normal mixtures, even for dimension as large as 20. To our knowledge, there exists no application of RJMCMC for such high-dimensional mixtures. Further, we propose a novel methodology to summarize the posterior, providing a way to obtain the mode of the posterior distribution of the densities and the associated highest posterior density credible regions. Based on our method we also propose a criterion to assess convergence of variable-dimensional algorithms. These methods of summarization and convergence assessment are applicable to general problems, not just to mixtures.
Modelling extremes using approximate Bayesian Computation

By the nature of their construction, many statistical models for extremes result in likelihood functions that are computationally prohibitive to evaluate. This is consequently problematic for the purposes of likelihood-based inference. With a focus on the Bayesian framework, this chapter examines the use of approximate Bayesian computation (ABC) techniques for the fitting and analysis of statistical models for extremes. After introducing the ideas behind ABC algorithms and methods, we demonstrate their application to extremal models in stereology and spatial extremes.
A fast Mixed Model B-splines algorithm

A fast algorithm for B-splines in mixed models is presented. B-splines have local support and are computational attractive, because the corresponding matrices are sparse. A key element of the new algorithm is that the local character of B-splines is preserved, while in other existing methods this local character is lost. The computation time for the fast algorithm is linear in the number of B-splines, while computation time scales cubically for existing transformations.
Extending the Square Root Method to Account for Additive Forecast Noise in Ensemble Methods

AbstractA square root approach is considered for the problem of accounting for model noise in the forecast step of the ensemble Kalman filter (EnKF) and related algorithms. The primary aim is to replace the method of simulated, pseudo-random additive so as to eliminate the associated sampling errors. The core method is based on the analysis step of ensemble square root filters, and consists in the deterministic computation of a transform matrix. The theoretical advantages regarding dynamical consistency are surveyed, applying equally well to the square root method in the analysis step. A fundamental problem due to the limited size of the ensemble subspace is discussed, and novel solutions that complement the core method are suggested and studied. Benchmarks from twin experiments with simple, low-order dynamics indicate improved performance over standard approaches such as additive, simulated noise, and multiplicative inflation.
A New Step-down Procedure for Simultaneous Hypothesis Testing Under Dependence

In this article, we consider the problem of simultaneous testing of hypotheses when the individual test statistics are not necessarily independent. Specifically, we consider the problem of simultaneous testing of point null hypotheses against two-sided alternatives for the mean parameters of normally distributed random variables. We assume that conditionally given the vector of means, these random variables jointly follow a multivariate normal distribution with a known but arbitrary covariance matrix. We consider a Bayesian framework where each unknown mean parameter is modeled through a two-component "spike and slab" mixture prior. This way, unconditionally the test statistics jointly have a mixture of multivariate normal distributions. A new testing procedure is developed that uses the dependence among the test statistics and works in a "step-down" manner. This procedure is general enough to be applied for non-normal data. A decision theoretic justification in favor of the proposed testing procedure has been provided by showing that unlike many traditional p-value based stepwise procedures, this new method possesses a certain "convexity property" which makes it admissible with respect to a vector risk function that captures the risks for the individual testing problems. An alternative representation of the proposed test statistics has also been established resulting in great simplification in the computational complexity. It is demonstrated through extensive simulations that for various forms of dependence and a wide range of sparsity levels, the proposed testing procedure compares quite favorably with several existing multiple testing procedures available in the literature in terms of overall misclassification probability.
Rao-Blackwellised Interacting Markov Chain Monte Carlo for Electromagnetic Scattering Inversion

The following electromagnetism (EM) inverse problem is addressed. It consists in estimating local radioelectric properties of materials recovering an object from the global EM scattering measurement, at various incidences and wave frequencies. This large scale ill-posed inverse problem is explored by an intensive exploitation of an efficient 2D Maxwell solver, distributed on High Performance Computing (HPC) machines. Applied to a large training data set, a statistical analysis reduces the problem to a simpler probabilistic metamodel, on which Bayesian inference can be performed. Considering the radioelectric properties as a dynamic stochastic process, evolving in function of the frequency, it is shown how advanced Markov Chain Monte Carlo methods, called Sequential Monte Carlo (SMC) or interacting particles, can provide estimations of the EM properties of each material, and their associated uncertainties.
Data Dependent Convergence for Distributed Stochastic Optimization

In this dissertation we propose alternative analysis of distributed stochastic gradient descent (SGD) algorithms that rely on spectral properties of the data covariance. As a consequence we can relate questions pertaining to speedups and convergence rates for distributed SGD to the data distribution instead of the regularity properties of the objective functions. More precisely we show that this rate depends on the spectral norm of the sample covariance matrix. An estimate of this norm can provide practitioners with guidance towards a potential gain in algorithm performance. For example many sparse datasets with low spectral norm prove to be amenable to gains in distributed settings. Towards establishing this data dependence we first study a distributed consensus-based SGD algorithm and show that the rate of convergence involves the spectral norm of the sample covariance matrix when the underlying data is assumed to be independent and identically distributed (homogenous). This dependence allows us to identify network regimes that prove to be beneficial for datasets with low sample covariance spectral norm. Existing consensus based analyses prove to be sub-optimal in the homogenous setting. Our analysis method also allows us to find data-dependent convergence rates as we limit the amount of communication. Spreading a fixed amount of data across more nodes slows convergence; in the asymptotic regime we show that adding more machines can help when minimizing twice-differentiable losses. Since the mini-batch results don't follow from the consensus results we propose a different data dependent analysis thereby providing theoretical validation for why certain datasets are more amenable to mini-batching. We also provide empirical evidence for results in this thesis.
A Hybrid Adaptive MCMC Algorithm in Function Spaces

The preconditioned Crank-Nicolson (pCN) method is a Markov Chain Monte Carlo (MCMC) scheme, specifically designed to perform Bayesian inferences in function spaces. Unlike many standard MCMC algorithms, the pCN method can preserve the sampling efficiency under the mesh refinement, a property referred to as being dimension independent. In this work we consider an adaptive strategy to further improve the efficiency of pCN. In particular we develop a hybrid adaptive MCMC method: the algorithm performs an adaptive Metropolis scheme in a chosen finite dimensional subspace, and a standard pCN algorithm in the complement space of the chosen subspace. We show that the proposed algorithm satisfies certain important ergodicity conditions. Finally with numerical examples we demonstrate that the proposed method has competitive performance with existing adaptive algorithms.
Quantized Nonparametric Estimation

We present an extension to Pinsker's theorem for nonparametric estimation over Sobolev ellipsoids when estimation is carried out under storage or communication constraints. Placing limits on the number of bits used to encode any estimator, we give tight lower and upper bounds on the excess risk due to quantization in terms of the number of bits, the signal size, and the noise level. This establishes the Pareto optimal minimax tradeoff between storage and risk under quantization constraints for Sobolev spaces. Our results and proof techniques combine elements of rate distortion theory and minimax analysis.
Faculty Opinions recommendation of Non-T cell activation linker (NTAL): a transmembrane adaptor protein involved in immunoreceptor signaling.

A key molecule necessary for activation of T lymphocytes through their antigen-specific T cell receptor (TCR) is the transmembrane adaptor protein LAT (linker for activation of T cells). Upon TCR engagement, LAT becomes rapidly tyrosine phosphorylated and then serves as a scaffold organizing a multicomponent complex that is indispensable for induction of further downstream steps of the signaling cascade. Here we describe the identification and preliminary characterization of a novel transmembrane adaptor protein that is structurally and evolutionarily related to LAT and is expressed in B lymphocytes, natural killer (NK) cells, monocytes, and mast cells but not in resting T lymphocytes. This novel transmembrane adaptor protein, termed NTAL (non–T cell activation linker) is the product of a previously identified WBSCR5 gene of so far unknown function. NTAL becomes rapidly tyrosine-phosphorylated upon cross-linking of the B cell receptor (BCR) or of high-affinity Fc (cid:2) - and Fc (cid:3) -receptors of myeloid cells and then associates with the cytoplasmic signaling molecules Grb2, Sos1, Gab1, and c-Cbl. NTAL expressed in the LAT-deficient T cell line J.CaM2. 5 becomes tyrosine phosphorylated and rescues activation of Erk1/2 and minimal transient elevation of cytoplasmic calcium level upon TCR/CD3 cross-linking. Thus, NTAL appears to be a structural and possibly also functional homologue of LAT in non–T cells. mAb) at 10 (cid:6) g/ml final concentration was added and the measurement was continued for additional 5 min. Finally, adequacy of cellular loading was verified by treating the cells with ionomycin (Sigma-Aldrich; 2 (cid:6) g/ml final concen-tration). Data were acquired on FACSort™ flow cytometer (Bec-ton Dickinson) at 500 ms time points and ratiometric analysis was performed with Flow Jo software (Tree Star).
Bayesian Models for Heterogeneous Personalized Health Data

The purpose of this study is to leverage modern technology (such as mobile or web apps in Beckman et al. (2014)) to enrich epidemiology data and infer the transmission of disease. Homogeneity related research on population level has been intensively studied in previous work. In contrast, we develop hierarchical Graph-Coupled Hidden Markov Models (hGCHMMs) to simultaneously track the spread of infection in a small cell phone community and capture person-specific infection parameters by leveraging a link prior that incorporates additional covariates. We also reexamine the model evolution of the hGCHMM from simple HMMs and LDA, elucidating additional flexibility and interpretability. Due to the non-conjugacy of sparsely coupled HMMs, we design a new approximate distribution, allowing our approach to be more applicable to other application areas. Additionally, we investigate two common link functions, the beta-exponential prior and sigmoid function, both of which allow the development of a principled Bayesian hierarchical framework for disease transmission. The results of our model allow us to predict the probability of infection for each person on each day, and also to infer personal physical vulnerability and the relevant association with covariates. We demonstrate our approach experimentally on both simulation data and real epidemiological records.
DPM: A State Space Model for Large-Scale Direct Marketing

We propose a novel statistical model to answer three challenges in direct marketing: which channel to use, which offer to make, and when to offer. There are several potential applications for the proposed model, for example, developing personalized marketing strategies and monitoring members' needs. Furthermore, the results from the model can complement and can be integrated with other existing models. The proposed model, named Dynamic Propensity Model, is a latent variable time series model that utilizes both marketing and purchase histories of a customer. The latent variable in the model represents the customer's propensity to buy a product. The propensity derives from purchases and other observable responses. Marketing touches increase a member's propensity, and propensity score attenuates and propagates over time as governed by data-driven parameters. To estimate the parameters of the model, a new statistical methodology has been developed. This methodology makes use of particle methods with a stochastic gradient descent approach, resulting in fast estimation of the model coefficients even from big datasets. The model is validated using six months' marketing records from one of the largest insurance companies in the U.S. Experimental results indicate that the effects of marketing touches vary depending on both channels and products. We compare the predictive performance of the proposed model with lagged variable logistic regression. Limitations and extensions of the proposed algorithm are also discussed.
A Brief Introduction to the Temporal Group LASSO and its Potential Applications in Healthcare

The Temporal Group LASSO is an example of a multi-task, regularized regression approach for the prediction of response variables that vary over time. The aim of this work is to introduce the reader to the concepts behind the Temporal Group LASSO and its related methods, as well as to the type of potential applications in a healthcare setting that the method has. We argue that the method is attractive because of its ability to reduce overfitting, select predictors, learn smooth effect patterns over time, and finally, its simplicity
Partition MCMC for Inference on Acyclic Digraphs

ABSTRACT Acyclic digraphs are the underlying representation of Bayesian networks, a widely used class of probabilistic graphical models. Learning the underlying graph from data is a way of gaining insights about the structural properties of a domain. Structure learning forms one of the inference challenges of statistical graphical models. Markov chain Monte Carlo (MCMC) methods, notably structure MCMC, to sample graphs from the posterior distribution given the data are probably the only viable option for Bayesian model averaging. Score modularity and restrictions on the number of parents of each node allow the graphs to be grouped into larger collections, which can be scored as a whole to improve the chain’s convergence. Current examples of algorithms taking advantage of grouping are the biased order MCMC, which acts on the alternative space of permuted triangular matrices, and nonergodic edge reversal moves. Here, we propose a novel algorithm, which employs the underlying combinatorial structure of DAGs to define a new grouping. As a result convergence is improved compared to structure MCMC, while still retaining the property of producing an unbiased sample. Finally, the method can be combined with edge reversal moves to improve the sampler further. Supplementary materials for this article are available online.