Miracle-dz/UniFAR-FTUs huggingface dataset:
Estimating minimum effect with outlier selection

We introduce one-sided versions of Huber's contamination model, in which corrupted samples tend to take larger values than uncorrupted ones. Two intertwined problems are addressed: estimation of the mean of uncorrupted samples (minimum effect) and selection of corrupted samples (outliers). Regarding the minimum effect estimation, we derive the minimax risks and introduce adaptive estimators to the unknown number of contaminations. Interestingly, the optimal convergence rate highly differs from that in classical Huber's contamination model. Also, our analysis uncovers the effect of particular structural assumptions on the distribution of the contaminated samples. As for the problem of selecting the outliers, we formulate the problem in a multiple testing framework for which the location/scaling of the null hypotheses are unknown. We rigorously prove how estimating the null hypothesis is possible while maintaining a theoretical guarantee on the amount of the falsely selected outliers, both through false discovery rate (FDR) or post hoc bounds. As a by-product, we address a long-standing open issue on FDR control under equi-correlation, which reinforces the interest of removing dependency when making multiple testing.
Interpolation error of misspecified Gaussian process regression

An interpolation error is an integral of the squared error of a regression model over a domain of interest. We consider the interpolation error for the case of misspecified Gaussian process regression: used covariance function differs from the true one. We derive the interpolation error for an infinite grid design of experiments. In particular, we show that for Matern 1/2 covariance function poor estimation of parameters only slightly affects the quality of interpolation. Then we proceed to numerical experiments that consider the misspecification for the most common covariance functions including other Matern and squared exponential covariance functions. For them, the quality of estimates of parameters affects the interpolation error.
Deep Representation with ReLU Neural Networks

We consider deep feedforward neural networks with rectified linear units from a signal processing perspective. In this view, such representations mark the transition from using a single (data-driven) linear representation to utilizing a large collection of affine linear representations tailored to particular regions of the signal space. This paper provides a precise description of the individual affine linear representations and corresponding domain regions that the (data-driven) neural network associates to each signal of the input space. In particular, we describe atomic decompositions of the representations and, based on estimating their Lipschitz regularity, suggest some conditions that can stabilize learning independent of the network depth. Such an analysis may promote further theoretical insight from both the signal processing and machine learning communities.
Causal Discovery Toolbox: Uncover causal relationships in Python

This paper presents a new open source Python framework for causal discovery from observational data and domain background knowledge, aimed at causal graph and causal mechanism modeling. The 'cdt' package implements the end-to-end approach, recovering the direct dependencies (the skeleton of the causal graph) and the causal relationships between variables. It includes algorithms from the 'Bnlearn' and 'Pcalg' packages, together with algorithms for pairwise causal discovery such as ANM. 'cdt' is available under the MIT License at this https URL.
Gaussian Process Mixtures for Estimating Heterogeneous Treatment Effects

We develop a Gaussian-process mixture model for heterogeneous treatment effect estimation that leverages the use of transformed outcomes. The approach we will present attempts to improve point estimation and uncertainty quantification relative to past work that has used transformed variable related methods as well as traditional outcome modeling. Earlier work on modeling treatment effect heterogeneity using transformed outcomes has relied on tree based methods such as single regression trees and random forests. Under the umbrella of non-parametric models, outcome modeling has been performed using Bayesian additive regression trees and various flavors of weighted single trees. These approaches work well when large samples are available, but suffer in smaller samples where results are more sensitive to model misspecification - our method attempts to garner improvements in inference quality via a correctly specified model rooted in Bayesian non-parametrics. Furthermore, while we begin with a model that assumes that the treatment assignment mechanism is known, an extension where it is learnt from the data is presented for applications to observational studies. Our approach is applied to simulated and real data to demonstrate our theorized improvements in inference with respect to two causal estimands: the conditional average treatment effect and the average treatment effect. By leveraging our correctly specified model, we are able to more accurately estimate the treatment effects while reducing their variance.
Learning formation energy of inorganic compounds using matrix variate deep Gaussian process

Future advancement of engineering applications is dependent on design of novel materials with desired properties. Enormous size of known chemical space necessitates use of automated high throughput screening to search the desired material. The high throughput screening uses quantum chemistry calculations to predict material properties, however, computational complexity of these calculations often imposes prohibitively high cost on the search for desired material. This critical bottleneck is resolved by using deep machine learning to emulate the quantum computations. However, the deep learning algorithms require a large training dataset to ensure an acceptable generalization, which is often unavailable a-priory. In this paper, we propose a deep Gaussian process based approach to develop an emulator for quantum calculations. We further propose a novel molecular descriptor that enables implementation of the proposed approach. As demonstrated in this paper, the proposed approach can be implemented using a small dataset. We demonstrate efficacy of our approach for prediction of formation energy of inorganic molecules.
Conformal symplectic and relativistic optimization

Arguably, the two most popular accelerated or momentum-based optimization methods in machine learning are Nesterov’s accelerated gradient and Polyaks’s heavy ball, both corresponding to different discretizations of a particular second order differential equation with friction. Such connections with continuous-time dynamical systems have been instrumental in demystifying acceleration phenomena in optimization. Here we study structure-preserving discretizations for a certain class of dissipative (conformal) Hamiltonian systems, allowing us to analyse the symplectic structure of both Nesterov and heavy ball, besides providing several new insights into these methods. Moreover, we propose a new algorithm based on a dissipative relativistic system that normalizes the momentum and may result in more stable/faster optimization. Importantly, such a method generalizes both Nesterov and heavy ball, each being recovered as distinct limiting cases, and has potential advantages at no additional cost.
Adaptation in multivariate log-concave density estimation

We study the adaptation properties of the multivariate log-concave maximum likelihood estimator over two subclasses of log-concave densities. The first consists of densities with polyhedral support whose logarithms are piecewise affine. The complexity of such densities $f$ can be measured in terms of the sum $\Gamma(f)$ of the numbers of facets of the subdomains in the polyhedral subdivision of the support induced by $f$. Given $n$ independent observations from a $d$-dimensional log-concave density with $d \in \{2,3\}$, we prove a sharp oracle inequality, which in particular implies that the Kullback--Leibler risk of the log-concave maximum likelihood estimator for such densities is bounded above by $\Gamma(f)/n$, up to a polylogarithmic factor. Thus, the rate can be essentially parametric, even in this multivariate setting. The second type of subclass consists of densities whose contours are well-separated; these new classes are constructed to be affine invariant and turn out to contain a wide variety of densities, including those that satisfy Holder regularity conditions. Here, we prove another sharp oracle inequality, which reveals in particular that the log-concave maximum likelihood estimator attains a Kullback--Leibler risk bound of order $n^{-\min\bigl(\frac{\beta+3}{\beta+7},\frac{4}{7}\bigr)}$ when $d=3$ over the class of $\beta$-Holder log-concave densities with $\beta > 1$, again up to a polylogarithmic factor.
High-dimensional joint estimation of multiple directed Gaussian graphical models

We consider the problem of jointly estimating multiple related directed acyclic graph (DAG) models based on high-dimensional data from each graph. This problem is motivated by the task of learning gene regulatory networks based on gene expression data from different tissues, developmental stages or disease states. We prove that under certain regularity conditions, the proposed $\ell_0$-penalized maximum likelihood estimator converges in Frobenius norm to the adjacency matrices consistent with the data-generating distributions and has the correct sparsity. In particular, we show that this joint estimation procedure leads to a faster convergence rate than estimating each DAG model separately. As a corollary, we also obtain high-dimensional consistency results for causal inference from a mix of observational and interventional data. For practical purposes, we propose \emph{jointGES} consisting of Greedy Equivalence Search (GES) to estimate the union of all DAG models followed by variable selection using lasso to obtain the different DAGs, and we analyze its consistency guarantees. The proposed method is illustrated through an analysis of simulated data as well as epithelial ovarian cancer gene expression data.
Accelerating delayed-acceptance Markov chain Monte Carlo algorithms

Delayed-acceptance Markov chain Monte Carlo (DA-MCMC) samples from a probability distribution via a two-stages version of the Metropolis-Hastings algorithm, by combining the target distribution with a "surrogate" (i.e. an approximate and computationally cheaper version) of said distribution. DA-MCMC accelerates MCMC sampling in complex applications, while still targeting the exact distribution. We design a computationally faster, albeit approximate, DA-MCMC algorithm. We consider parameter inference in a Bayesian setting where a surrogate likelihood function is introduced in the delayed-acceptance scheme. When the evaluation of the likelihood function is computationally intensive, our scheme produces a 2-4 times speed-up, compared to standard DA-MCMC. However, the acceleration is highly problem dependent. Inference results for the standard delayed-acceptance algorithm and our approximated version are similar, indicating that our algorithm can return reliable Bayesian inference. As a computationally intensive case study, we introduce a novel stochastic differential equation model for protein folding data.
Adaptive thresholding estimation of a Poisson intensity with infinite support

The purpose of this paper is to estimate the intensity of a Poisson process $N$ by using thresholding rules. In this paper, the intensity, defined as the derivative of the mean measure of $N$ with respect to $ndx$ where $n$ is a fixed parameter, is assumed to be non-compactly supported. The estimator $\tilde{f}_{n,\gamma}$ based on random thresholds is proved to achieve the same performance as the oracle estimator up to a logarithmic term. Oracle inequalities allow to derive the maxiset of $\tilde{f}_{n,\gamma}$. Then, minimax properties of $\tilde{f}_{n,\gamma}$ are established. We first prove that the rate of this estimator on Besov spaces ${\cal B}^\al_{p,q}$ when $p\leq 2$ is $(\ln(n)/n)^{\al/(1+2\al)}$. This result has two consequences. First, it establishes that the minimax rate of Besov spaces ${\cal B}^\al_{p,q}$ with $p\leq 2$ when non compactly supported functions are considered is the same as for compactly supported functions up to a logarithmic term. This result is new. Furthermore, $\tilde{f}_{n,\gamma}$ is adaptive minimax up to a logarithmic term. When $p>2$, the situation changes dramatically and the rate of $\tilde{f}_{n,\gamma}$ on Besov spaces ${\cal B}^\al_{p,q}$ is worse than $(\ln(n)/n)^{\al/(1+2\al)}$. Finally, the random threshold depends on a parameter $\gamma$ that has to be suitably chosen in practice. Some theoretical results provide upper and lower bounds of $\gamma$ to obtain satisfying oracle inequalities. Simulations reinforce these results.
Optimal Sparse Singular Value Decomposition for High-Dimensional High-Order Data

Abstract In this article, we consider the sparse tensor singular value decomposition, which aims for dimension reduction on high-dimensional high-order data with certain sparsity structure. A method named sparse tensor alternating thresholding for singular value decomposition (STAT-SVD) is proposed. The proposed procedure features a novel double projection & thresholding scheme, which provides a sharp criterion for thresholding in each iteration. Compared with regular tensor SVD model, STAT-SVD permits more robust estimation under weaker assumptions. Both the upper and lower bounds for estimation accuracy are developed. The proposed procedure is shown to be minimax rate-optimal in a general class of situations. Simulation studies show that STAT-SVD performs well under a variety of configurations. We also illustrate the merits of the proposed procedure on a longitudinal tensor dataset on European country mortality rates. Supplementary materials for this article are available online.
A Unified Framework for Low-Rank plus Sparse Matrix Recovery

We propose a unified framework to solve general low-rank plus sparse matrix recovery problems based on matrix factorization, which covers a broad family of objective functions satisfying the restricted strong convexity and smoothness conditions. Based on projected gradient descent and the double thresholding operator, our proposed generic algorithm is guaranteed to converge to the unknown low-rank and sparse matrices at a locally linear rate, while matching the best-known robustness guarantee (i.e., tolerance for sparsity). At the core of our theory is a novel structural Lipschitz gradient condition for low-rank plus sparse matrices, which is essential for proving the linear convergence rate of our algorithm, and we believe is of independent interest to prove fast rates for general superposition-structured models. We illustrate the application of our framework through two concrete examples: robust matrix sensing and robust PCA. Experiments on both synthetic and real datasets corroborate our theory.
Examining the Effects of Objective Hurricane Risks and Community Resilience on Risk Perceptions of Hurricanes at the County Level in the U.S. Gulf Coast: An Innovative Approach

Communities' risk perceptions can influence their abilities to cope with coastal hazards such as hurricanes and coastal flooding. Our study presents an initial effort to examine the relationship between community resilience and risk perception at the county level, through innovative construction of aggregate variables. Using the 2012 Gulf Coast Climate Change Survey merged with historical hurricane data and community resilience indicators, we first apply a spatial statistical model to construct a county-level risk perception indicator based on survey responses. Next, we employ regression to reveal the relationship between contextual hurricane risk factors and community resilience on one hand and county-level perceptions of hurricane risks on the other. Results of this study are directly applicable in the policymaking domain as many hazard mitigation plans and adaptation policies are designed and implemented at the county level. Specifically, two major findings stand out. First, the contextual hurricane risks represented by peak height of storm surge associated with the last hurricane landfall and land area exposed to historical storm surge flooding positively affect county-level risk perceptions. This indicates that another threat of hurricanes—wind risks—needs to be clearly communicated with the public and fully incorporated into hazard mitigation plans and adaptation policies. Second, two components of community resilience—higher levels of economic resilience and community capital—are found to lead to heightened perceptions of hurricane risks, which suggests that concerted efforts are needed to raise awareness of hurricane risks among counties with less economic and community capital.
Asymptotic analysis of selection-mutation models in the presence of multiple fitness peaks

We study the long-time behaviour of phenotype-structured models describing the evolutionary dynamics of asexual populations whose phenotypic fitness landscape is characterised by multiple peaks. First we consider the case where phenotypic changes do not occur, and then we include the effect of heritable phenotypic changes. In the former case the model is formulated as an integrodifferential equation for the phenotype distribution of the individuals in the population, whereas in the latter case the evolution of the phenotype distribution is governed by a non-local parabolic equation whereby a linear diffusion operator captures the presence of phenotypic changes. We prove that the long-time limit of the solution to the integrodifferential equation is unique and given by a measure consisting of a weighted sum of Dirac masses centred at the peaks of the phenotypic fitness landscape. We also derive an explicit formula to compute the weights in front of the Dirac masses. Moreover, we demonstrate that the long-time solution of the non-local parabolic equation exhibits a qualitatively similar behaviour in the asymptotic regime where the diffusion coefficient modelling the rate of phenotypic change tends to zero. However, we show that the limit measure of the non-local parabolic equation may consist of less Dirac masses, and we provide a sufficient criterion to identify the positions of their centres. Finally, we carry out a detailed characterisation of the speed of convergence of the integral of the solution (i.e. the population size) to its long-time limit for both models. Taken together, our results support a more in-depth theoretical understanding of the conditions leading to the emergence of stable phenotypic polymorphism in asexual populations.
Inversion Copulas from Nonlinear State Space Models

We propose to construct copulas from the inversion of nonlinear state space models. These allow for new time series models that have the same serial dependence structure of a state space model, but with an arbitrary marginal distribution, and flexible density forecasts. We examine the time series properties of the copulas, outline serial dependence measures, and estimate the models using likelihood-based methods. Copulas constructed from three example state space models are considered: a stochastic volatility model with an unobserved component, a Markov switching autoregression, and a Gaussian linear unobserved component model. We show that all three inversion copulas with flexible margins improve the fit and density forecasts of quarterly U.S. broad inflation and electricity inflation.
Efficient data augmentation for multivariate probit models with panel data: an application to general practitioner decision making about contraceptives

The paper considers the problem of estimating a multivariate probit model in a panel data setting with emphasis on sampling a high dimensional correlation matrix and improving the overall efficiency of the data augmentation approach. We reparameterize the correlation matrix in a principled way and then carry out efficient Bayesian inference by using Hamiltonian Monte Carlo sampling. We also propose a novel antithetic variable method to generate samples from the posterior distribution of the random effects and regression coefficients, resulting in significant gains in efficiency. We apply the methodology by analysing stated preference data obtained from Australian general practitioners evaluating alternative contraceptive products. Our analysis suggests that the joint probability of discussing combinations of contraceptive products with a patient shows medical practice variation among the general practitioners, which indicates some resistance even to discuss these products, let alone to recommend them.
Algebra and geometry of tensors for modeling rater agreement data

We study three different quasi-symmetry models and three different mixture models of $n\times n\times n$ tensors for modeling rater agreement data. For these models we give a geometric description of the associated varieties and we study their invariants distinguishing between the case $n=2$ and the case $n>2$. Finally, for the two models for pairwise agreement we state some results about the pairwise Cohen's $\kappa$ coefficients.
A Bayesian Covariance Graph and Latent Position Model for Multivariate Financial Time Series

Current understanding holds that financial contagion is driven mainly by system-wide interconnectedness of institutions. A distinction has been made between systematic and idiosyncratic channels of contagion, with shocks transmitted through the latter expected to be substantially more likely to lead to a crisis than through the former. Idiosyncratic connectivity is thought to be driven not simply by obviously shared characteristics among institutions, but more by the latent strategic position of ? rms in ? nancial markets. We propose a Bayesian hierarchical model for multivariate ? nancial time series that characterizes the interdependence in the idiosyncratic factors of a VAR model via a covariance graphical model whose structure is modeled through a latent position model. We develop an efficient algorithm that samples the network of the idiosyncratic factors and the latent positions underlying the network. We examine the dynamic volatility network and latent positions among 150 publicly listed institutions across the United States and Europe and how they contribute to systemic vulnerabilities and risk transmission.
Statistical Failure Mechanism Analysis of Earthquakes Revealing Time Relationships

If we assume that earthquakes are chaotic, and influenced locally then chaos theory suggests that there should be a temporal association between earthquakes in a local region that should be revealed with statistical examination. To date no strong relationship has been shown (refs not prediction). However, earthquakes are basically failures of structured material systems, and when multiple failure mechanisms are present, prediction of failure is strongly inhibited without first separating the mechanisms. Here we show that by separating earthquakes statistically, based on their central tensor moment structure, along lines first suggested by a separation into mechanisms according to depth of the earthquake, a strong indication of temporal association appears. We show this in earthquakes above 200 Km along the pacific ring of fire, with a positive association in time between earthquakes of the same statistical type and a negative association in time between earthquakes of different types. Whether this can reveal either useful mechanistic information to seismologists, or can result in useful forecasts remains to be seen.