Miracle-dz/UniFAR-FTUs huggingface dataset:
Optimal Policies for Observing Time Series and Related Restless Bandit Problems

The trade-off between the cost of acquiring and processing data, and uncertainty due to a lack of data is fundamental in machine learning. A basic instance of this trade-off is the problem of deciding when to make noisy and costly observations of a discrete-time Gaussian random walk, so as to minimise the posterior variance plus observation costs. We present the first proof that a simple policy, which observes when the posterior variance exceeds a threshold, is optimal for this problem. The proof generalises to a wide range of cost functions other than the posterior variance. This result implies that optimal policies for linear-quadratic-Gaussian control with costly observations have a threshold structure. It also implies that the restless bandit problem of observing multiple such time series, has a well-defined Whittle index. We discuss computation of that index, give closed-form formulae for it, and compare the performance of the associated index policy with heuristic policies. The proof is based on a new verification theorem that demonstrates threshold structure for Markov decision processes, and on the relation between binary sequences known as mechanical words and the dynamics of discontinuous nonlinear maps, which frequently arise in physics, control and biology.
Sparse Estimation of Generalized Linear Models (GLM) via Approximated Information Criteria

We propose a new sparse estimation method, termed MIC (Minimum approximated Information Criterion), for generalized linear models (GLM) in fixed dimensions. What is essentially involved in MIC is the approximation of the $\ell_0$-norm with a continuous unit dent function. Besides, a reparameterization step is devised to enforce sparsity in parameter estimates while maintaining the smoothness of the objective function. MIC yields superior performance in sparse estimation by optimizing the approximated information criterion without reducing the search space and is computationally advantageous since no selection of tuning parameters is required. Moreover, the reparameterization tactic leads to valid significance testing results that are free of post-selection inference. We explore the asymptotic properties of MIC and illustrate its usage with both simulated experiments and empirical examples.
Non-separable covariance models for spatio-temporal data, with applications to neural encoding analysis

Neural encoding studies explore the relationships between measurements of neural activity and measurements of a behavior that is viewed as a response to that activity. The coupling between neural and behavioral measurements is typically imperfect and difficult to measure.To enhance our ability to understand neural encoding relationships, we propose that a behavioral measurement may be decomposable as a sum of two latent components, such that the direct neural influence and prediction is primarily localized to the component which encodes temporal dependence. For this purpose, we propose to use a non-separable Kronecker sum covariance model to characterize the behavioral data as the sum of terms with exclusively trial-wise, and exclusively temporal dependencies. We then utilize a corrected form of Lasso regression in combination with the nodewise regression approach for estimating the conditional independence relationships between and among variables for each component of the behavioral data, where normality is necessarily assumed. We provide the rate of convergence for estimating the precision matrices associated with the temporal as well as spatial components in the Kronecker sum model. We illustrate our methods and theory using simulated data, and data from a neural encoding study of hawkmoth flight; we demonstrate that the neural encoding signal for hawkmoth wing strokes is primarily localized to a latent component with temporal dependence, which is partially obscured by a second component with trial-wise dependencies.
Penalized Maximum Tangent Likelihood Estimation and Robust Variable Selection

We introduce a new class of mean regression estimators -- penalized maximum tangent likelihood estimation -- for high-dimensional regression estimation and variable selection. We first explain the motivations for the key ingredient, maximum tangent likelihood estimation (MTE), and establish its asymptotic properties. We further propose a penalized MTE for variable selection and show that it is $\sqrt{n}$-consistent, enjoys the oracle property. The proposed class of estimators consists penalized $\ell_2$ distance, penalized exponential squared loss, penalized least trimmed square and penalized least square as special cases and can be regarded as a mixture of minimum Kullback-Leibler distance estimation and minimum $\ell_2$ distance estimation. Furthermore, we consider the proposed class of estimators under the high-dimensional setting when the number of variables $d$ can grow exponentially with the sample size $n$, and show that the entire class of estimators (including the aforementioned special cases) can achieve the optimal rate of convergence in the order of $\sqrt{\ln(d)/n}$. Finally, simulation studies and real data analysis demonstrate the advantages of the penalized MTE.
Comparison between continuous and discrete doses using Escalation With Overdose Control

Although there is an extensive statistical literature showing the disadvantages of discretizing continuous variables, categorization is a common practice in clinical research which results in substantial loss of information. A large collection of methods in cancer phase I clinical trial design establishes dose of a new agent as a discrete variable. A noteworthy exception is the Escalation With Overdose Control (EWOC) design, where doses can be defined either as continuous or as a grid of discrete doses. A Monte Carlo simulation study was performed to compare the operating characteristics of continuous and discrete dose EWOC designs. Four equally spaced grids with different interval lengths were considered. The loss of information was measured by several operating characteristics easier for clinicians to interpret, in addition to the usual statistical measures of bias and mean squared error. Based on the simulations, if there is not enough knowledge about the true MTD value as commonly happens in phase I clinical trials, continuous dose scheme arises as an attractive option.
A group-based approach to the least squares regression for handling multicollinearity from strongly correlated variables

Multicollinearity due to strongly correlated predictor variables is a long-standing problem in regression analysis. It leads to difficulties in parameter estimation, inference, variable selection and prediction for the least squares regression. To deal with these difficulties, we propose a group-based approach to the least squares regression centered on the collective impact of the strongly correlated variables. We discuss group effects of such variables that represent their collective impact, and present the group-based approach through real and simulated data examples. We also give a condition more precise than what is available in the literature under which predictions by the least squares estimated model are accurate. This approach is a natural way of working with multicollinearity which resolves the difficulties without altering the least squares method. It has several advantages over alternative methods such as ridge regression and principal component regression.
Comparative Causal Mediation: Relaxing the Assumption of No Mediator-Outcome Confounding

Experiments often include multiple treatments, with the primary goal to compare the causal effects of multiple treatments. While comparing the magnitudes of the average treatment effects (ATEs) is straightforward, there exist few methods to systematically compare the causal anatomies of each treatment (that is, the collection of causal mechanisms underlying each treatment's total effect) in order to understand the sources of their relative magnitudes. This study introduces a framework for comparing the causal anatomies of multiple treatments through the use of causal mediation analysis. The study proposes a set of comparative causal mediation estimands that compare the mediation effects of different treatments via a common mediator. It derives the properties of a set of estimators, which are shown to be consistent (or conservative) without making the assumption of no unobserved confounding of the mediator-outcome relationship, which is a strong and nonrefutable assumption that must be made for consistent estimation of individual causal mediation effects. The estimators are easy to understand and implement, thereby providing researchers with a simple, reliable, and systematic method of comparing, discovering, and testing the causal mechanism differences between multiple treatments. An original application is presented to illustrate the method.
On the geometric ergodicity of Hamiltonian Monte Carlo

We establish general conditions under which Markov chains produced by the Hamiltonian Monte Carlo method will and will not be geometrically ergodic. We consider implementations with both position-independent and position-dependent integration times. In the former case we find that the conditions for geometric ergodicity are essentially a gradient of the log-density which asymptotically points towards the centre of the space and grows no faster than linearly. In an idealised scenario in which the integration time is allowed to change in different regions of the space, we show that geometric ergodicity can be recovered for a much broader class of tail behaviours, leading to some guidelines for the choice of this free parameter in practice.
Combining Random Walks and Nonparametric Bayesian Topic Model for Community Detection

Community detection has been an active research area for decades. Among all probabilistic models, Stochastic Block Model has been the most popular one. This paper introduces a novel probabilistic model: RW-HDP, based on random walks and Hierarchical Dirichlet Process, for community extraction. In RW-HDP, random walks conducted in a social network are treated as documents; nodes are treated as words. By using Hierarchical Dirichlet Process, a nonparametric Bayesian model, we are not only able to cluster nodes into different communities, but also determine the number of communities automatically. We use Stochastic Variational Inference for our model inference, which makes our method time efficient and can be easily extended to an online learning algorithm.
Beyond correlation in spatial statistics modeling

We introduce a model for spatial statistics which can account explicitly for interactions among more than two field components at a time. The theoretical aspects of the model are dealt with: cumulant and moment generating functions, spatial consistency and parameter estimation. On the basis of a detailed synthetic example, we show the kind of inference about the (partially observed) spatial field that can be very wrong, if one validates his model by checking only one and two dimensional marginal fit, and covariance function fit. We suggest statistics that can be used additionally for model validation, which help assess interdependence among groups of variables. The implications of considering multivariate interactions for intense daily precipitation forecasting over a small catchment in southeastern Germany (that of the Saalach river) are investigated.
Inference of weighted $V$-statistics for nonstationary time series and its applications

We investigate the behavior of Fourier transforms for a wide class of nonstationary nonlinear processes. Asymptotic central and noncentral limit theorems are established for a class of nondegenerate and degenerate weighted $V$-statistics through the angle of Fourier analysis. The established theory for $V$-statistics provides a unified treatment for many important time and spectral domain problems in the analysis of nonstationary time series, ranging from nonparametric estimation to the inference of periodograms and spectral densities.
Consistent Nonparametric Different-Feature Selection via the Sparsest $k$-Subgraph Problem

Two-sample feature selection is the problem of finding features that describe a difference between two probability distributions, which is a ubiquitous problem in both scientific and engineering studies. However, existing methods have limited applicability because of their restrictive assumptions on data distributoins or computational difficulty. In this paper, we resolve these difficulties by formulating the problem as a sparsest $k$-subgraph problem. The proposed method is nonparametric and does not assume any specific parametric models on the data distributions. We show that the proposed method is computationally efficient and does not require any extra computation for model selection. Moreover, we prove that the proposed method provides a consistent estimator of features under mild conditions. Our experimental results show that the proposed method outperforms the current method with regard to both accuracy and computation time.
Machine Learning Meets Microeconomics: The Case of Decision Trees and Discrete Choice

We provide a microeconomic framework for decision trees: a popular machine learning method. Specifically, we show how decision trees represent a non-compensatory decision protocol known as disjunctions-of-conjunctions and how this protocol generalizes many of the non-compensatory rules used in the discrete choice literature so far. Additionally, we show how existing decision tree variants address many economic concerns that choice modelers might have. Beyond theoretical interpretations, we contribute to the existing literature of two-stage, semi-compensatory modeling and to the existing decision tree literature. In particular, we formulate the first bayesian model tree, thereby allowing for uncertainty in the estimated non-compensatory rules as well as for context-dependent preference heterogeneity in one's second-stage choice model. Using an application of bicycle mode choice in the San Francisco Bay Area, we estimate our bayesian model tree, and we find that it is over 1,000 times more likely to be closer to the true data-generating process than a multinomial logit model (MNL). Qualitatively, our bayesian model tree automatically finds the effect of bicycle infrastructure investment to be moderated by travel distance, socio-demographics and topography, and our model identifies diminishing returns from bike lane investments. These qualitative differences lead to bayesian model tree forecasts that directly align with the observed bicycle mode shares in regions with abundant bicycle infrastructure such as Davis, CA and the Netherlands. In comparison, MNL's forecasts are overly optimistic.
Mixture Modeling on Related Samples by ψ-Stick Breaking and Kernel Perturbation

There has been great interest recently in applying nonparametric kernel mixtures in a hierarchical manner to model multiple related data samples jointly. In such settings several data features are commonly present: (i) the related samples often share some, if not all, of the mixture components but with differing weights, (ii) only some, not all, of the mixture components vary across the samples, and (iii) often the shared mixture components across samples are not aligned perfectly in terms of their location and spread, but rather display small misalignments either due to systematic cross-sample difference or more often due to uncontrolled, extraneous causes. Properly incorporating these features in mixture modeling will enhance the efficiency of inference, whereas ignoring them not only reduces efficiency but can jeopardize the validity of the inference due to issues such as confounding. We introduce two techniques for incorporating these features in modeling related data samples using kernel mixtures. The first technique, called $\psi$-stick breaking, is a joint generative process for the mixing weights through the breaking of both a stick shared by all the samples for the components that do not vary in size across samples and an idiosyncratic stick for each sample for those components that do vary in size. The second technique is to imbue random perturbation into the kernels, thereby accounting for cross-sample misalignment. These techniques can be used either separately or together in both parametric and nonparametric kernel mixtures. We derive efficient Bayesian inference recipes based on MCMC sampling for models featuring these techniques, and illustrate their work through both simulated data and a real flow cytometry data set in prediction/estimation, cross-sample calibration, and testing multi-sample differences.
Kernel Two-Sample Hypothesis Testing Using Kernel Set Classification

The two-sample hypothesis testing problem is studied for the challenging scenario of high dimensional data sets with small sample sizes. We show that the two-sample hypothesis testing problem can be posed as a one-class set classification problem. In the set classification problem the goal is to classify a set of data points that are assumed to have a common class. We prove that the average probability of error given a set is less than or equal to the Bayes error and decreases as a power of $n$ number of sample data points in the set. We use the positive definite Set Kernel for directly mapping sets of data to an associated Reproducing Kernel Hilbert Space, without the need to learn a probability distribution. We specifically solve the two-sample hypothesis testing problem using a one-class SVM in conjunction with the proposed Set Kernel. We compare the proposed method with the Maximum Mean Discrepancy, F-Test and T-Test methods on a number of challenging simulated high dimensional and small sample size data. We also perform two-sample hypothesis testing experiments on six cancer gene expression data sets and achieve zero type-I and type-II error results on all data sets.
Monitoring Chinese Population Migration in Consecutive Weekly Basis from Intra-city scale to Inter-province scale by Didi's Bigdata

Population migration is valuable information which leads to proper decision in urban-planning strategy, massive investment, and many other fields. For instance, inter-city migration is a posterior evidence to see if the government's constrain of population works, and inter-community immigration might be a prior evidence of real estate price hike. With timely data, it is also impossible to compare which city is more favorable for the people, suppose the cities release different new regulations, we could also compare the customers of different real estate development groups, where they come from, where they probably will go. Unfortunately these data was not available. In this paper, leveraging the data generated by positioning team in Didi, we propose a novel approach that timely monitoring population migration from community scale to provincial scale. Migration can be detected as soon as in a week. It could be faster, the setting of a week is for statistical purpose. A monitoring system is developed, then applied nation wide in China, some observations derived from the system will be presented in this paper. This new method of migration perception is origin from the insight that nowadays people mostly moving with their personal Access Point (AP), also known as WiFi hotspot. Assume that the ratio of AP moving to the migration of population is constant, analysis of comparative population migration would be feasible. More exact quantitative research would also be done with few sample research and model regression. The procedures of processing data includes many steps: eliminating the impact of pseudo-migration AP, for instance pocket WiFi, and second-hand traded router; distinguishing moving of population with moving of companies; identifying shifting of AP by the finger print clusters, etc.
Fused Gaussian Process for Very Large Spatial Data

With the development of new remote sensing technology, large or even massive spatial datasets covering the globe becomes available. Statistical analysis of such data is challenging. This article proposes a semiparametric approach to model large or massive spatial datasets. In particular, a Gaussian process with additive components is proposed, with its covariance structure consisting of two components: one component is flexible without assuming a specific parametric covariance function but is able to achieve dimension reduction; the other is parametric and simultaneously induces sparsity. The inference algorithm for parameter estimation and spatial prediction is devised. The resulting spatial prediction method that we call fused Gaussian process (FGP), is applied to simulated data and a massive satellite dataset. The results demonstrate the computational and inferential benefits of the FGP over competing methods and show that the FGP is more flexible and robust against model misspecification.
Lasso Meets Horseshoe: A Survey

The goal of this paper is to contrast and survey the major advances in two of the most commonly used high-dimensional techniques, namely, the Lasso and horseshoe regularization. Lasso is a gold standard for predictor selection while horseshoe is a state-of-the-art Bayesian estimator for sparse signals. Lasso is fast and scalable and uses convex optimization whilst the horseshoe is non-convex. Our novel perspective focuses on three aspects: (i) theoretical optimality in high dimensional inference for the Gaussian sparse model and beyond, (ii) efficiency and scalability of computation and (iii) methodological development and performance.
Functional Inference on Rotational Curves and Identification of Human Gait at the Knee Joint

We extend Gaussian perturbation models in classical functional data analysis to the three-dimensional rotational group where a zero-mean Gaussian process in the Lie algebra under the Lie exponential spreads multiplicatively around a central curve. As an estimator, we introduce point-wise extrinsic mean curves which feature strong perturbation consistency, and which are asymptotically a.s. unique and differentiable, if the model is so. Further, we consider the group action of time warping and that of spatial isometries that are connected to the identity. The latter can be asymptotically consistently estimated if lifted to the unit quaternions. Introducing a generic loss for Lie groups, the former can be estimated, and based on curve length, due to asymptotic differentiability, we propose two-sample permutation tests involving various combinations of the group actions. This methodology allows inference on gait patterns due to the rotational motion of the lower leg with respect to the upper leg. This was previously not possible because, among others, the usual analysis of separate Euler angles is not independent of marker placement, even if performed by trained specialists.
Relating Diversity and Human Appropriation from Land Cover Data

We present a method to describe the relation between indicators of landscape diversity and the human appropriation of the net primary production in a given region. These quantities are viewed as functions of the vector of proportions of the different land covers, which is in turn treated as a random vector whose values depend on the particular small terrain cell that is observed. We illustrate the method assuming first that the vector of proportions follows a uniform distribution on the simplex. We then consider as starting point a raw dataset of observed proportions for each cell, for which we must first obtain an estimate of its theoretical probability distribution, and secondly generate a sample of large size from it. We apply this procedure to real historical data of the Mallorca Island in three different moments of time. Our main goal is to compute the mean value of the landscape diversity as a function of the level of human appropriation. This function is related to the so-called Energy-Species hypothesis and to the Intermediate Disturbance Hypothesis.