Miracle-dz/UniFAR-FTUs huggingface dataset:
Low Rank Independence Samplers in Bayesian Inverse Problems

In Bayesian inverse problems, the posterior distribution is used to quantify uncertainty about the reconstructed solution. In practice, Markov chain Monte Carlo algorithms often are used to draw samples from the posterior distribution. However, implementations of such algorithms can be computationally expensive. We present a computationally efficient scheme for sampling high-dimensional Gaussian distributions in ill-posed Bayesian linear inverse problems. Our approach uses Metropolis-Hastings independence sampling with a proposal distribution based on a low-rank approximation of the prior-preconditioned Hessian. We show the dependence of the acceptance rate on the number of eigenvalues retained and discuss conditions under which the acceptance rate is high. We demonstrate our proposed sampler by using it with Metropolis-Hastings-within-Gibbs sampling in numerical experiments in image deblurring, computerized tomography, and NMR relaxometry.
Binary sampling from discrete distributions

This paper considers direct sampling methods from discrete target distributions. The inverse transform sampling (ITS) method is one of the most popular direct sampling methods. The main purpose of this paper is to propose a direct sampling algorithm that supersedes the binary-search ITS method (which is an improvement of the ITS method with binary search). The proposed algorithm is based on binarizing the support set of the target distribution. Thus, the proposed algorithm is referred to as binary sampling (BS). The BS algorithm consists of two procedures: backward binary sampling (BBS) and forward binary sampling (FBS). The BBS procedure draws a single sample (the first sample) from the target distribution while constructing a one-way random walk on a binary tree for the FBS procedure. By running the random walk, the FBS procedure generates the second and subsequent samples. The BBS and FBS procedures have $O(N)$ and $O(\ln N)$ time complexities, respectively, and they also have $O(N)$ space complexity, where $N+1$ is the cardinality of the support set of the target distribution. Therefore, the time and space complexities of the BS algorithm are equivalent to those of the standard (possibly best) binary-search ITS algorithm. However, the BS algorithm has two advantages over the standard binary-search ITS algorithm. First, the BBS procedure is parallelizable and thus the total running time of the BS algorithm can be reduced. Second, the BS algorithm is more accurate in terms of relative rounding error that influences generated samples.
Reciprocal Graphical Models for Integrative Gene Regulatory Network Analysis

Constructing gene regulatory networks is a fundamental task in systems biology. We introduce a Gaussian reciprocal graphical model for inference about gene regulatory relationships by integrating mRNA gene expression and DNA level information including copy number and methylation. Data integration allows for inference on the directionality of certain regulatory relationships, which would be otherwise indistinguishable due to Markov equivalence. Efficient inference is developed based on simultaneous equation models. Bayesian model selection techniques are adopted to estimate the graph structure. We illustrate our approach by simulations and two applications in ZODIAC pairwise gene interaction analysis and colon adenocarcinoma pathway analysis.
Estimation and Model Identification of Locally Stationary Varying-Coefficient Additive Models

Nonparametric regression models with locally stationary covariates have received increasing interest in recent years. As a nice relief of "curse of dimensionality" induced by large dimension of covariates, additive regression model is commonly used. However, in locally stationary context, to catch the dynamic nature of regression function, we adopt a flexible varying-coefficient additive model where the regression function has the form $\alpha_{0}\left(u\right)+\sum_{k=1}^{p}\alpha_{k}\left(u\right)\beta_{k}\left(x_{k}\right). $ For this model, we propose a three-step spline estimation method for each univariate nonparametric function, and show its consistency and $L_{2}$ rate of convergence. Furthermore, based upon the three-step estimators, we develop a two-stage penalty procedure to identify pure additive terms and varying-coefficient terms in varying-coefficient additive model. As expected, we demonstrate that the proposed identification procedure is consistent, and the penalized estimators achieve the same $L_{2}$ rate of convergence as the polynomial spline estimators. Simulation studies are presented to illustrate the finite sample performance of the proposed three-step spline estimation method and two-stage model selection procedure.
An M-Estimator for Reduced-Rank High-Dimensional Linear Dynamical System Identification

High-dimensional time-series data are becoming increasingly abundant across a wide variety of domains, spanning economics, neuroscience, particle physics, and cosmology. Fitting statistical models to such data, to enable parameter estimation and time-series prediction, is an important computational primitive. Existing methods, however, are unable to cope with the high-dimensional nature of these problems, due to both computational and statistical reasons. We mitigate both kinds of issues via proposing an M-estimator for Reduced-rank System IDentification (MR. SID). A combination of low-rank approximations, L-1 and L-2 penalties, and some numerical linear algebra tricks, yields an estimator that is computationally efficient and numerically stable. Simulations and real data examples demonstrate the utility of this approach in a variety of problems. In particular, we demonstrate that MR. SID can estimate spatial filters, connectivity graphs, and time-courses from native resolution functional magnetic resonance imaging data. Other applications and extensions are immediately available, as our approach is a generalization of the classical Kalman Filter-Smoother Expectation-Maximization algorithm.
Understanding Betting Strategy

In this paper, we present betting strategy of a football game using probability theory. We know all betting houses offer slightly unfair odds towards the player. Here we discuss a simple way to figure out which betting house is offering relatively better odds compared to others for English Premier League. However, this methodology can be adopted for another league football match.
Random networks, graphical models and exchangeability

We study conditional independence relationships for random networks and their interplay with exchangeability. We show that, for finitely exchangeable network models, the empirical subgraph densities are maximum likelihood estimates of their theoretical counterparts. We then characterize all possible Markov structures for finitely exchangeable random graphs, thereby identifying a new class of Markov network models corresponding to bidirected Kneser graphs. In particular, we demonstrate that the fundamental property of dissociatedness corresponds to a Markov property for exchangeable networks described by bidirected line graphs. Finally we study those exchangeable models that are also summarized in the sense that the probability of a network depends only on the degree distribution, and we identify a class of models that is dual to the Markov graphs of Frank and Strauss. Particular emphasis is placed on studying consistency properties of network models under the process of forming subnetworks and we show that the only consistent systems of Markov properties correspond to the empty graph, the bidirected line graph of the complete graph and the complete graph.
An Inexact Variable Metric Proximal Point Algorithm for Generic Quasi-Newton Acceleration

We propose an inexact variable-metric proximal point algorithm to accelerate gradient-based optimization algorithms. The proposed scheme, called QNing can be notably applied to incremental first-order methods such as the stochastic variance-reduced gradient descent algorithm (SVRG) and other randomized incremental optimization algorithms. QNing is also compatible with composite objectives, meaning that it has the ability to provide exactly sparse solutions when the objective involves a sparsity-inducing regularization. When combined with limited-memory BFGS rules, QNing is particularly effective to solve high-dimensional optimization problems, while enjoying a worst-case linear convergence rate for strongly convex problems. We present experimental results where QNing gives significant improvements over competing methods for training machine learning methods on large samples and in high dimensions.
Approximate Marginal Posterior for Log Gaussian Cox Processes

The log Gaussian Cox process is a flexible class of Cox processes, whose intensity surface is stochastic, for incorporating complex spatial and time structure of point patterns. The straightforward inference based on Markov chain Monte Carlo is computationally heavy because the computational cost of inverse or Cholesky decomposition of high dimensional covariance matrices of Gaussian latent variables is cubic order of their dimension. Furthermore, since hyperparameters for Gaussian latent variables have high correlations with sampled Gaussian latent processes themselves, standard Markov chain Monte Carlo strategies are inefficient. In this paper, we propose an efficient and scalable computational strategy for spatial log Gaussian Cox processes. The proposed algorithm is based on pseudo-marginal Markov chain Monte Carlo approach. Based on this approach, we propose estimation of approximate marginal posterior for parameters and comprehensive model validation strategies. We provide details for all of the above along with some simulation investigation for univariate and multivariate settings and analysis of a point pattern of tree data exhibiting positive and negative interaction between different species.
Analyzing Two-Stage Experiments in the Presence of Interference

ABSTRACT Two-stage randomization is a powerful design for estimating treatment effects in the presence of interference; that is, when one individual’s treatment assignment affects another individual’s outcomes. Our motivating example is a two-stage randomized trial evaluating an intervention to reduce student absenteeism in the School District of Philadelphia. In that experiment, households with multiple students were first assigned to treatment or control; then, in treated households, one student was randomly assigned to treatment. Using this example, we highlight key considerations for analyzing two-stage experiments in practice. Our first contribution is to address additional complexities that arise when household sizes vary; in this case, researchers must decide between assigning equal weight to households or equal weight to individuals. We propose unbiased estimators for a broad class of individual- and household-weighted estimands, with corresponding theoretical and estimated variances. Our second contribution is to connect two common approaches for analyzing two-stage designs: linear regression and randomization inference. We show that, with suitably chosen standard errors, these two approaches yield identical point and variance estimates, which is somewhat surprising given the complex randomization scheme. Finally, we explore options for incorporating covariates to improve precision. We confirm our analytic results via simulation studies and apply these methods to the attendance study, finding substantively meaningful spillover effects.
Effects of associated kernels in nonparametric multiple regressions

Associated kernels have been introduced to improve the classical continuous kernels for smoothing any function on several kinds of supports, such as bounded continuous and discrete sets. This work deals with the effects of combined associated kernels on nonpara-metric multiple regression functions. Using the Nadaraya-Watson estimator with optimal bandwidth matrices selected by a cross-validation procedure, different behaviors of multiple regression estimations are pointed out according to the type of multivariate associated kernels with correlation or not. Through simulation studies, there are no effects of correlation structures for the continuous regression functions and also for the associated continuous kernels; however, there exist real effects of the choice of multivariate associated kernels following the support of the multiple regression functions, whether bounded continuous or discrete. Applications are made on two real data sets.
Fractional compound Poisson processes with multiple internal states

For the particles undergoing the anomalous diffusion with different waiting time distributions for different internal states, we derive the Fokker-Planck and Feymann-Kac equations, respectively, describing positions of the particles and functional distributions of the trajectories of particles; in particular, the equations governing the functional distribution of internal states are also obtained. The dynamics of the stochastic processes are analyzed and the applications, calculating the distribution of the first passage time and the distribution of the fraction of the occupation time, of the equations are given.
Incoherence of Partial-Component Sampling in multidimensional NMR

In NMR spectroscopy, undersampling in the indirect dimensions causes reconstruction artifacts whose size can be bounded using the so-called {\it coherence}. In experiments with multiple indirect dimensions, new undersampling approaches were recently proposed: random phase detection (RPD) \cite{Maciejewski11} and its generalization, partial component sampling (PCS) \cite{Schuyler13}. The new approaches are fully aware of the fact that high-dimensional experiments generate hypercomplex-valued free induction decays; they randomly acquire only certain low-dimensional components of each high-dimensional hypercomplex entry. We provide a classification of various hypercomplex-aware undersampling schemes, and define a hypercomplex-aware coherence appropriate for such undersampling schemes; we then use it to quantify undersampling artifacts of RPD and various PCS schemes.
Semiparametric panel data models using neural networks

This paper presents an estimator for semiparametric models that uses a feed-forward neural network to fit the nonparametric component. Unlike many methodologies from the machine learning literature, this approach is suitable for longitudinal/panel data. It provides unbiased estimation of the parametric component of the model, with associated confidence intervals that have near-nominal coverage rates. Simulations demonstrate (1) efficiency, (2) that parametric estimates are unbiased, and (3) coverage properties of estimated intervals. An application section demonstrates the method by predicting county-level corn yield using daily weather data from the period 1981-2015, along with parametric time trends representing technological change. The method is shown to out-perform linear methods such as OLS and ridge/lasso, as well as random forest. The procedures described in this paper are implemented in the R package panelNNET.
Model Selection for Explosive Models

This paper examines the limit properties of information criteria for distinguishing between the unit root model and the various kinds of explosive models. The information criteria include AIC, BIC, HQIC. The explosive models include the local-to-unit-root model, the mildly explosive model and the regular explosive model. Initial conditions with different order of magnitude are considered. Both the OLS estimator and the indirect inference estimator are studied. It is found that BIC and HQIC, but not AIC, consistently select the unit root model when data come from the unit root model. When data come from the local-to-unit-root model, both BIC and HQIC select the wrong model with probability approaching 1 while AIC has a positive probability of selecting the right model in the limit. When data come from the regular explosive model or from the mildly explosive model in the form of 1+n^{\alpha}/n with \alpha \in (0; 1), all three information criteria consistently select the true model. Indirect inference estimation can increase or decrease the probability for information criteria to select the right model asymptotically relative to OLS, depending on the information criteria and the true model. Simulation results confirm our asymptotic results in finite sample.
On a consistent estimator of a useful signal in Ornstein-Uhlenbeck model in $\mathbb{C}[-l,l[$

~It is considered a transmittion process of a useful signal in Ornstein-Uhlenbeck model in $\mathbb{C}[-l,l[$ defined by the stochastic differential equation $$ d\Psi(t,x,\omega)=\sum_{n=0}^{2m} A_n\frac{\partial^{n}}{\partial x^{n}}\Psi(t,x,\omega)dt +\sigma d W(t,\omega) $$ with initial condition $$\Psi(0,x,\omega)=\Psi_0(x) \in FD^{(0)}[-l,l[, $$ where $m \ge 1$, $(A_n)_{0 \le n \le 2m} \in \mathbb{R}^+\times \mathbb{R}^{2m-1}$,$~((t,x,\omega) \in [0,+\infty[\times [-l,l[ \times \Omega)$, $\sigma \in \mathbb{R}^+$, $\mathbb{C}[-l,l[$ is Banach space of all real-valued bounded continuous functions on $[-l,l[$, $FD^{(0)}[-l,l[ \subset \mathbb{C}[-l,l[ $ is class of all real-valued bounded continuous functions on $[-l,l[$ whose Fourier series converges to himself everywhere on $[-l,l[$, $(W(t,\omega))_{t \ge 0}$ is a Wiener process and $\Psi_0(x)$ is a useful signal. By use a sequence of transformed signals $(Z_k)_{k \in N}=(\Psi(t_0,x,\omega_k))_{k \in N}$ at moment $t_0>0$, consistent and infinite-sample consistent estimations of the useful signal $\Psi_0$ is constructed under assumption that parameters $(A_n)_{0 \le n \le 2m}$ and $\sigma$ are known. Animation and simulation of the Ornstein-Uhlenbeck process in $\mathbb{C}[-l,l[$ and an estimation of a useful signal are also presented.
Multivariate extensions of expectiles risk measures

Abstract This paper is devoted to the introduction and study of a new family of multivariate elicitable risk measures. We call the obtained vector-valued measures multivariate expectiles. We present the different approaches used to construct our measures. We discuss the coherence properties of these multivariate expectiles. Furthermore, we propose a stochastic approximation tool of these risk measures.
Consistency Guarantees for Permutation-Based Causal Inference Algorithms

Directed acyclic graph (DAG) models, are widely used to represent complex causal systems. Since the basic task of learning a DAG model from data is NP-hard, a standard approach is greedy search over the space of DAGs or Markov equivalence classes of DAGs. Since the space of DAGs on $p$ nodes and the associated space of Markov equivalence classes are both much larger than the space of permutations, it is desirable to consider permutation-based greedy searches. We here provide the first consistency guarantees, both uniform and high-dimensional, of a greedy permutation-based search. This search corresponds to a simplex-type algorithm over the edge-graph of a sub-polytope of the permutohedron, called the DAG associahedron. Every vertex in this polytope is associated with a DAG, and hence with a collection of permutations that are consistent with the DAG ordering. A walk is performed on the edges of the polytope maximizing the sparsity of the associated DAGs. We also show based on simulations that this permutation search is competitive with current approaches.
Learning the Sparse and Low Rank PARAFAC Decomposition via the Elastic Net

In this article, we derive a Bayesian model to learning the sparse and low rank PARAFAC decomposition for the observed tensor with missing values via the elastic net, with property to find the true rank and sparse factor matrix which is robust to the noise. We formulate efficient block coordinate descent algorithm and admax stochastic block coordinate descent algorithm to solve it, which can be used to solve the large scale problem. To choose the appropriate rank and sparsity in PARAFAC decomposition, we will give a solution path by gradually increasing the regularization to increase the sparsity and decrease the rank. When we find the sparse structure of the factor matrix, we can fixed the sparse structure, using a small to regularization to decreasing the recovery error, and one can choose the proper decomposition from the solution path with sufficient sparse factor matrix with low recovery error. We test the power of our algorithm on the simulation data and real data, which show it is powerful.
Directional Metropolis-Hastings

We propose a new kernel for Metropolis Hastings called Directional Metropolis Hastings (DMH) with multivariate update where the proposal kernel has state dependent covariance matrix. We use the derivative of the target distribution at the current state to change the orientation of the proposal distribution, therefore producing a more plausible proposal. We study the conditions for geometric ergodicity of our algorithm and provide necessary and sufficient conditions for convergence. We also suggest a scheme for adaptively update the variance parameter and study the conditions of ergodicity of the adaptive algorithm. We demonstrate the performance of our algorithm in a Bayesian generalized linear model problem.
A Unified Parallel Algorithm for Regularized Group PLS Scalable to Big Data

Partial Least Squares (PLS) methods have been heavily exploited to analyse the association between two blocs of data. These powerful approaches can be applied to data sets where the number of variables is greater than the number of observations and in presence of high collinearity between variables. Different sparse versions of PLS have been developed to integrate multiple data sets while simultaneously selecting the contributing variables. Sparse modelling is a key factor in obtaining better estimators and identifying associations between multiple data sets. The cornerstone of the sparsity version of PLS methods is the link between the SVD of a matrix (constructed from deflated versions of the original matrices of data) and least squares minimisation in linear regression. We present here an accurate description of the most popular PLS methods, alongside their mathematical proofs. A unified algorithm is proposed to perform all four types of PLS including their regularised versions. Various approaches to decrease the computation time are offered, and we show how the whole procedure can be scalable to big data sets.
A dynamic stochastic blockmodel for interaction lengths

We propose a new dynamic stochastic blockmodel that focuses on the analysis of interaction lengths in networks. The model does not rely on a discretization of the time dimension and may be used to analyze networks that evolve continuously over time. The framework relies on a clustering structure on the nodes, whereby two nodes belonging to the same latent group tend to create interactions and non-interactions of similar lengths. We introduce a fast variational expectation-maximization algorithm to perform inference, and adapt a widely used clustering criterion to perform model choice. Finally, we test our methodology on artificial data, and propose a demonstration on a dataset concerning face-to-face interactions between students in a high-school.
Strong Consistency of Multivariate Spectral Variance Estimators

Markov chain Monte Carlo (MCMC) algorithms are used to estimate features of interest of a distribution. The Monte Carlo error in estimation has an asymptotic normal distribution whose multivariate nature has so far been ignored in the MCMC community. We present a class of multivariate spectral variance estimators for the asymptotic covariance matrix in the Markov chain central limit theorem and provide conditions for strong consistency. We examine the finite sample properties of the multivariate spectral variance estimators and its eigenvalues in the context of a vector autoregressive process of order 1.