Miracle-dz/UniFAR-FTUs huggingface dataset:
Shannon's entropy and its Generalizations towards Statistics, Reliability and Information Science during 1948-2018

Starting from the pioneering works of Shannon and Weiner in 1948, a plethora of works have been reported on entropy in different directions. Entropy-related review work in the direction of statistics, reliability and information science, to the best of our knowledge, has not been reported so far. Here we have tried to collect all possible works in this direction during the period 1948-2018 so that people interested in entropy, specially the new researchers, get benefited.
Variational Joint Filtering

New technologies for recording the activity of large neural populations during complex behavior provide exciting opportunities for investigating the neural computations that underlie perception, cognition, and decision-making. Nonlinear state-space models provide an interpretable signal processing framework by combining an intuitive dynamical system with a probabilistic observation model, which can provide insights into neural dynamics, neural computation, and development of neural prosthetics and treatment through feedback control. It yet brings the challenge of learning both latent neural state and the underlying dynamical system because neither is known for neural systems a priori. We developed a flexible online learning framework for latent nonlinear state dynamics and filtered latent states. Using the stochastic gradient variational Bayes approach, our method jointly optimizes the parameters of the nonlinear dynamical system, the observation model, and the black-box recognition model. Unlike previous approaches, our framework can incorporate non-trivial distributions of observation noise and has constant time and space complexity. These features make our approach amenable to real-time applications and the potential to automate analysis and experimental design in ways that testably track and modify behavior using stimuli designed to influence learning.
Modeling the spatio-temporal dynamics of land use change with recurrent neural networks.

This study applies recurrent neural networks (RNNs), which are known for its ability to process sequential information, to model the spatio-temporal dynamics of land use change (LUC) and to forecast annual land use maps of the city of Tsukuba, Japan. We develop two categories of RNN models: 1) simple RNN, which is the basic RNN variant, 2) three RNN variants with advanced gated architecture: long short-term memory (LSTM), LSTM with peephole connection (LSTM-peephole), and gated recurrent unit (GRU) models. The four models are developed using spatio-temporal data with high temporal resolution, annual data for the periods 2000 to 2010, 2011 and 2012 to 2016 are used for training, validation and testing, respectively. The predictive performances are evaluated using classification metrics (accuracy and F1 score) and the map comparison metrics (Kappa simulation and fuzzy Kappa simulation). The results show that all RNN models achieve F1 scores higher than 0.55, and Kappa simulations higher than 0.47. Out of the four RNN models, LSTM and LSTM-peephole models significantly outperform the other two RNN models. Furthermore, LSTM-peephole model slightly outperforms the LSTM model. In addition, the results indicate that the RNN models with gated architecture, which have better ability to model longer temporal dependency, significantly outperform the simple RNN model. Moreover, the predictive performance of LSTM-peephole model gradually decreases with the decrease of temporal sequential length of the training set. These results demonstrate the benefit of taking temporal dependency into account to model the LUC process with RNNs.
Estimating the size of a hidden finite set: Large-sample behavior of estimators

A finite set is "hidden" if its elements are not directly enumerable or if its size cannot be ascertained via a deterministic query. In public health, epidemiology, demography, ecology and intelligence analysis, researchers have developed a wide variety of indirect statistical approaches, under different models for sampling and observation, for estimating the size of a hidden set. Some methods make use of random sampling with known or estimable sampling probabilities, and others make structural assumptions about relationships (e.g. ordering or network information) between the elements that comprise the hidden set. In this review, we describe models and methods for learning about the size of a hidden finite set, with special attention to asymptotic properties of estimators. We study the properties of these methods under two asymptotic regimes, "infill" in which the number of fixed-size samples increases, but the population size remains constant, and "outfill" in which the sample size and population size grow together. Statistical properties under these two regimes can be dramatically different.
Robust and Sparse Regression in GLM by Stochastic Optimization.

The generalized linear model (GLM) plays a key role in regression analyses. In high-dimensional data, the sparse GLM has been used but it is not robust against outliers. Recently, the robust methods have been proposed for the specific example of the sparse GLM. Among them, we focus on the robust and sparse linear regression based on the $\gamma$-divergence. The estimator of the $\gamma$-divergence has strong robustness under heavy contamination. In this paper, we extend the robust and sparse linear regression based on the $\gamma$-divergence to the robust and sparse GLM based on the $\gamma$-divergence with a stochastic optimization approach in order to obtain the estimate. We adopt the randomized stochastic projected gradient descent as a stochastic optimization approach and extend the established convergence property to the classical first-order necessary condition. By virtue of the stochastic optimization approach, we can efficiently estimate parameters for very large problems. Particularly, we show the linear regression, logistic regression and Poisson regression with $L_1$ regularization in detail as specific examples of robust and sparse GLM. In numerical experiments and real data analysis, the proposed method outperformed comparative methods.
Bayesian Modeling of Nonlinear Poisson Regression with Artificial Neural Networks

Being in the era of big data, modeling and prediction of count data have become significantly important in many fields including health, finance, social, etc. Although linear Poisson regression has been widely used to model count and rate data, it might not be always suitable as it cannot capture some inherent variability within complex data. In this study, we introduce a probabilistically driven nonlinear Poisson regression model with Bayesian artificial neural networks (ANN) to model count or rate data. This new nonlinear Poisson regression model developed with Bayesian ANN provides higher prediction accuracies over traditional Poisson or negative binomial regression models as revealed in our simulation and real data studies.
Supplementary Materials for ‘ Large-Scale Multiple Hypothesis Testing with the Normal-Beta Prime Prior ’

In Section A, we provide all the proofs of the main results. In Section B, we provide the technical details for implementing the empirical Bayes and fully Bayes variants of the NBP model. In Section C, we compare the posterior shrinkage weights to the theoretical posterior inclusion probabilities. Appendix A. Proofs of Main Results A.1. Proofs for Section 2.1 Proof of Proposition 2.1. As noted by Proposition 2.1 in [1], the beta prime density (7) can be rewritten as a product of independent gamma and inverse gamma densities. We thus reparametrize model (6) for a single observation θ as follows: θ|λiξ ∼ N (0, λξ), λ ∼ G(a, 1), ξ ∼ IG(b, 1). (A1) From (A1), we see that the joint distribution of the prior is proportional to π(θ, λ, ξ) ∝ (λξ)−1/2 exp ( − θ 2 2λξ ) λa−1 exp (−λ) exp ( − ξ ) ξ−b−1 = λa−3/2 exp(−λ)ξ−b−3/2 exp ( − ( θ2 2λ + 1 ) 1 ξ ) . CONTACT Ray Bai. Email: Ray.Bai@pennmedicine.upenn. edu
A detailed treatment of Doob's theorem

Doob's theorem provides guarantees of consistent estimation and posterior consistency under very general conditions. Despite the limitation that it only guarantees consistency on a set with prior probability 1, for many models arising in practice, Doob's theorem is an easy way of showing that consistency will hold almost everywhere. In this article, we give a detailed proof of Doob's theorem.
The Landscape of the Spiked Tensor Model

We consider the problem of estimating a large rank‐one tensor u⊗k ∈ (ℝn)⊗k, k ≥ 3, in Gaussian noise. Earlier work characterized a critical signal‐to‐noise ratio λ Bayes = O(1) above which an ideal estimator achieves strictly positive correlation with the unknown vector of interest. Remarkably, no polynomial‐time algorithm is known that achieved this goal unless λ ≥ Cn(k − 2)/4, and even powerful semidefinite programming relaxations appear to fail for 1 ≪ λ ≪ n(k − 2)/4.
CHARACTERIZATION OF THE TAIL BEHAVIOR OF A CLASS OF BEKK PROCESSES: A STOCHASTIC RECURRENCE EQUATION APPROACH

We consider conditions for strict stationarity and ergodicity of a class of multivariate BEKK processes $(X_t : t=1,2,\ldots )$ and study the tail behavior of the associated stationary distributions. Specifically, we consider a class of BEKK-ARCH processes where the innovations are assumed to be Gaussian and a finite number of lagged $X_t$ ’s may load into the conditional covariance matrix of $X_t$ . By exploiting that the processes have multivariate stochastic recurrence equation representations, we show the existence of strictly stationary solutions under mild conditions, where only a fractional moment of $X_t$ may be finite. Moreover, we show that each component of the BEKK processes is regularly varying with some tail index. In general, the tail index differs along the components, which contrasts with most of the existing literature on the tail behavior of multivariate GARCH processes. Lastly, in an empirical illustration of our theoretical results, we quantify the model-implied tail index of the daily returns on two cryptocurrencies.
Gaussian Generalized Tetranacci Numbers

In this paper, we dene Gaussian generalized Tetranacci numbers and as special cases, we investigate Gaussian Tetranacci and Gaussian Tetranacci-Lucas numbers with their properties. We present Binet's formulas, generating functions, and the summation formulas for Gaussian generalized Tetranacci numbers.Moreover, we give some identities connecting Gaussian Tetranacci and Gaussian Tetranacci-Lucas numbers. Furthermore, we present matrix formulation of Gaussian generalized Tetranacci numbers.
Économétrie & Machine Learning

L'econometrie et l'apprentissage machine semblent avoir une finalite en commun: construire un modele predictif, pour une variable d'interet, a l'aide de variables explicatives (ou features). Pourtant, ces deux champs se sont developpes en parallele, creant ainsi deux cultures differentes, pour paraphraser Breiman (2001a). Le premier visait a construire des modeles probabilistes permettant de decrire des phenomemes economiques. Le second utilise des algorithmes qui vont apprendre de leurs erreurs, dans le but, le plus souvent de classer (des sons, des images, etc). Or recemment, les modeles d'apprentissage se sont montres plus efficaces que les techniques econometriques traditionnelles (avec comme prix a payer un moindre pouvoir explicatif), et surtout, ils arrivent a gerer des donnees beaucoup plus volumineuses. Dans ce contexte, il devient necessaire que les econometres comprennent ce que sont ces deux cultures, ce qui les oppose et surtout ce qui les rapproche, afin de s'approprier des outils developpes par la communaute de l'apprentissage statistique, pour les integrer dans des modeles econometriques.
T-optimal design for multivariate polynomial regression using semidefinite programming

We consider T-optimal experiment design problems for discriminating multivariate polynomial regression models where the design space is defined by polynomial inequalities and the regression parameters are constrained to given convex sets. The original optimality criterion is reformulated as a convex optimization problem with a moment cone constraint. In the case of univariate regression models, an exact semidefinite representation of the moment cone constraint can be applied to obtain an equivalent semidefinite program. For general multivariate cases, we apply a moment relaxation technique and approximate the moment cone constraint by a hierarchy of semidefinite-representable outer approximations. When the relaxation hierarchy converges, the optimal discrimination design can be recovered from the optimal moment matrix, and its optimality confirmed by an equivalence theorem. The methodology is illustrated with several examples.
Bayesian Markov Switching Tensor Regression For Time-Varying Networks

We propose a new Bayesian Markov switching regression model for multi-dimensional arrays (tensors) of binary time series. We assume a zero-inflated logit dynamics with time-varying parameters and apply it to multi-layer temporal networks. The original contribution is threefold. First, in order to avoid over-fitting we propose a parsimonious parametrization of the model, based on a low-rank decomposition of the tensor of regression coefficients. Second, the parameters of the tensor model are driven by a hidden Markov chain, thus allowing for structural changes. The regimes are identified through prior constraints on the mixing probability of the zero-inflated model. Finally, we model the jointly dynamics of the network and of a set of variables of interest. We follow a Bayesian approach to inference, exploiting the Polya-Gamma data augmentation scheme for logit models in order to provide an efficient Gibbs sampler for posterior approximation. We show the effectiveness of the sampler on simulated datasets of medium-big sizes, finally we apply the methodology to a real dataset of financial networks.
Zero-Modified Poisson-Lindley distribution with applications in zero-inflated and zero-deflated count data

The main object of this article is to present an extension of the zero-inflated Poisson-Lindley distribution, called of zero-modified Poisson-Lindley. The additional parameter $\pi$ of the zero-modified Poisson-Lindley has a natural interpretation in terms of either zero-deflated/inflated proportion. Inference is dealt with by using the likelihood approach. In particular the maximum likelihood estimators of the distribution's parameter are compared in small and large samples. We also consider an alternative bias-correction mechanism based on Efron's bootstrap resampling. The model is applied to real data sets and found to perform better than other competing models.
A generalized multivariate Student-t mixture model for Bayesian classification and clustering of radar waveforms

In this paper, a generalized multivariate Student-t mixture model is developed for classification and clustering of Low Probability of Intercept radar waveforms. A Low Probability of Intercept radar signal is characterized by a pulse compression waveform which is either frequency-modulated or phase-modulated. The proposed model can classify and cluster different modulation types such as linear frequency modulation, non linear frequency modulation, polyphase Barker, polyphase P1, P2, P3, P4, Frank and Zadoff codes. The classification method focuses on the introduction of a new prior distribution for the model hyper-parameters that gives us the possibility to handle sensitivity of mixture models to initialization and to allow a less restrictive modeling of data. Inference is processed through a Variational Bayes method and a Bayesian treatment is adopted for model learning, supervised classification and clustering. Moreover, the novel prior distribution is not a well-known probability distribution and both deterministic and stochastic methods are employed to estimate its expectations. Some numerical experiments show that the proposed method is less sensitive to initialization and provides more accurate results than the previous state of the art mixture models.
Intelligent sampling for multiple change-points in exceedingly long time series with rate guarantees

Change point estimation in its offline version is traditionally performed by optimizing over the data set of interest, by considering each data point as the true location parameter and computing a data fit criterion. Subsequently, the data point that minimizes the criterion is declared as the change point estimate. For estimating multiple change points, the procedures are analogous in spirit, but significantly more involved in execution. Since change-points are local discontinuities, only data points close to the actual change point provide useful information for estimation, while data points far away are superfluous, to the point where using only a few points close to the true parameter is just as precise as using the full data set. Leveraging this "locality principle", we introduce a two-stage procedure for the problem at hand, which in the 1st stage uses a sparse subsample to obtain pilot estimates of the underlying change points, and in the 2nd stage refines these estimates by sampling densely in appropriately defined neighborhoods around them. We establish that this method achieves the same rate of convergence and even virtually the same asymptotic distribution as the analysis of the full data, while reducing computational complexity to O(N^0.5) time (N being the length of data set), as opposed to at least O(N) time for all current procedures, making it promising for the analysis on exceedingly long data sets with adequately spaced out change points. The main results are established under a signal plus noise model with independent and identically distributed error terms, but extensions to dependent data settings, as well as multiple stage (>2) procedures are also provided. The performance of our procedure -- which is coined "intelligent sampling" -- is illustrated on both synthetic and real Internet data streams.
Robust inference on indirect causal effects

Standard methods for inference about direct and indirect effects require stringent no unmeasured confounding assumptions which often fail to hold in practice, particularly in observational studies. The goal of this paper is to introduce a new form of indirect effect, the population intervention indirect effect (PIIE), that can be nonparametrically identified in the presence of an unmeasured common cause of exposure and outcome. This new type of indirect effect captures the extent to which the effect of exposure is mediated by an intermediate variable under an intervention which fixes the component of exposure directly influencing the outcome at its observed value. The PIIE is in fact the indirect component of the population intervention effect, introduced by Hubbard and Van der Laan (2008). Interestingly, our identification criterion relaxes Judea Pearl's front-door criterion as it does not require no direct effect of exposure not mediated by the intermediate variable. For inference, we develop both parametric and semiparametric methods, including a doubly robust semiparametric locally efficient estimator, that perform very well in simulation studies. Finally, the proposed methods are used to measure the effectiveness of monetary saving recommendations among women enrolled in a maternal health program in Tanzania.
A Source-Oriented Approach to Coal Power Plant Emissions Health Effects

There is increasing focus on whether air pollution originating from different sources has different health implications. In particular, recent evidence suggests that fine particulate matter (PM2.5) with chemical tracers suggesting coal combustion origins is especially harmful. Augmenting this knowledge with estimates from causal inference methods to identify the health impacts of PM2.5 derived from specific point sources of coal combustion would be an important step towards informing specific, targeted interventions. We investigated the effect of high-exposure to coal combustion emissions from 783 coal-fired power generating units on ischemic heart disease (IHD) hospitalizations in over 19 million Medicare beneficiaries residing at 21,351 ZIP codes in the eastern United States. We used InMAP, a newly-developed, reduced-complexity air quality model to classify each ZIP code as either a high-exposed or control location. Our health outcomes analysis uses a causal inference method - propensity score matching - to adjust for potential confounders of the relationship between exposure and IHD. We fit separate Poisson regression models to the matched data in each geographic region to estimate the incidence rate ratio for IHD comparing high-exposed to control locations. High exposure to coal power plant emissions and IHD were positively associated in the Northeast (IRR = 1.08, 95% CI = 1.06, 1.09) and the Southeast (IRR = 1.06, 95% CI = 1.04, 1.08). No significant association was found in the Industrial Midwest (IRR = 1.02, 95% CI = 1.00, 1.04), likely the result of small exposure contrasts between high-exposed and control ZIP codes in that region. This study provides targeted evidence of the association between emissions from specific coal power plants and IHD hospitalizations among Medicare beneficiaries.
Cox regression analysis with missing covariates via multiple imputation

We consider the situation of estimating Cox regression in which some covariates are subject to missing, and there exists additional information (including observed event time, censoring indicator and fully observed covariates) which may be predictive of the missing covariates. We propose to use two working regression models: one for predicting the missing covariates and the other for predicting the missing probabilities. For each missing covariate observation, these two working models are used to define a nearest neighbor imputing set. This set is then used to nonparametrically impute covariate values for the missing observation. Upon the completion of imputation, Cox regression is performed on the multiply imputed datasets to estimate the regression coefficients. In a simulation study, we compare the nonparametric multiple imputation approach with the augmented inverse probability weighted (AIPW) method, which directly incorporates the two working models into estimation of Cox regression. We show that all approaches can reduce bias due to non-ignorable missing mechanism. The proposed nonparametric imputation method is robust to mis-specification of either one of the two working models and robust to mis-specification of the link function of the two working models. In contrast, the AIPW method is not robust to mis-specification of the link function of the two working models and is sensitive to the selection probability. We apply the approaches to a breast cancer dataset from Surveillance, Epidemiology and End Results (SEER) Program .
Testing Exchangeability With Martingale for Change-Point Detection

This work studies the exchangeability test for a random sequence through a martingale based approach. Its main contributions include: 1) an additive martingale is introduced, which is more amenable for designing exchangeability tests by exploiting the Hoeffding-Azuma lemma; 2) different betting functions for constructing the additive martingale are studied. By choosing the underlying probability density function of p-values as betting function, it can be shown that, when change-point appears, a satisfying trade-off between the smoothness and expected one-step increment of the martingale sequence can be obtained. An online algorithm based on Beta distribution parametrization for constructing this betting function is discussed as well.