Miracle-dz/UniFAR-FTUs huggingface dataset:
Vertex nomination schemes for membership prediction

Suppose that a graph is realized from a stochastic block model where one of the blocks is of interest, but many or all of the vertices’ block labels are unobserved. The task is to order the vertices with unobserved block labels into a “nomination list” such that, with high probability, vertices from the interesting block are concentrated near the list’s beginning. We propose several vertex nomination schemes. Our basic—but principled—setting and development yields a best nomination scheme (which is a Bayes–Optimal analogue), and also a likelihood maximization nomination scheme that is practical to implement when there are a thousand vertices, and which is empirically near-optimal when the number of vertices is small enough to allow comparison to the best nomination scheme. We then illustrate the robustness of the likelihood maximization nomination scheme to the modeling challenges inherent in real data, using examples which include a social network involving human trafficking, the EnronGraph, a worm brain connectome and a political blog network. 1. Article overview. In a stochastic block model, the vertices of the graph are partitioned into blocks, and the existence/nonexistence of an edge between any pair of vertices is an independent Bernoulli trial, with the Bernoulli parameter being a function of the block memberships of the pair of vertices. We are concerned here with a graph realized from a stochastic block model such that many or all of the vertices’ block labels are hidden (i.e., unobserved). Suppose that one particular block is of interest, and the task is to order the vertices with a hidden block label into a “nomination list” with the goal of having vertices from the interesting block concentrated near
Smooth projected density estimation

We introduce and analyse a new nonparametric estimator of a multi-dimensional density. Our smooth projection estimator (SPE) is defined by a least squares projection of the sample onto an infinite dimensional mixture class via an undersmoothed nonparametric pilot estimate, which acts as a structural filter to regularise the solution. The undersmoothing is required to optimise the convergence rate of the SPE, which is jointly determined by that of the pilot estimator to the true density in squared $\mathbb{L}_{2}$ norm, and by that of the pilot distribution function to the empirical distribution function in uniform norm. Our procedure was conceived with a view to exploiting well known results in convex analysis and their connection to mixture densities. In the context of our work, this translates to the observation that the infinite dimensional minimisation problem, implicit in the construction of the SPE, possesses a solution of dimension at most $n+1$, where $n$ is the sample size. The SPE thus enjoys practical advantages such as computational efficiency, ease of storage and rapid evaluation at a new data point.
The factor paradox: Common factors can be correlated with the variance not accounted for by the common factors!

The case that the factor model does not account for all the covariances of the observed variables is considered. This is a quite realistic condition because some model error as well as some sampling error should usually occur with empirical data. It is shown that principal components representing covariances not accounted for by the factors of the model can have a non-zero correlation with the common factors of the factor model. Non-zero correlations of components representing variance not accounted for by the factor model with common factors were also found in a simulation study. Based on these results it should be concluded that common factors can be correlated with variance components representing model error as well as sampling error. In consequence, even when researchers decide not to represent some small or trivial variance by means of a common factor, these excluded variances can still be part of the model.
Variational Bayes inference and Dirichlet process priors

This paper shows how the variational Bayes method provides a computational efficient technique in the context of hierarchical modelling using Dirichlet process priors, in particular without requiring conjugate prior assumption. It shows, using the so called parameter separation parameterization, a simple criterion under which the variational method works well. Based on this framework, its provides a full variational solution for the Dirichlet process. The numerical results show that the method is very computationally efficient when compared to MCMC. Finally, we propose an empirical method to estimate the truncation level for the truncated Dirichlet process.
Likelihood Adaptively Modified Penalties

A new family of penalty functions, adaptive to likelihood, is introduced for model selection in general regression models. It arises naturally through assuming certain types of prior distribution on the regression parameters. To study stability properties of the penalized maximum likelihood estimator, two types of asymptotic stability are defined. Theoretical properties, including the parameter estimation consistency, model selection consistency, and asymptotic stability, are established under suitable regularity conditions. An efficient coordinate-descent algorithm is proposed. Simulation results and real data analysis show that the proposed method has competitive performance in comparison with existing ones.
Bayesian bandwidth estimation for a functional nonparametric regression model with mixed types of regressors and unknown error density

We investigate the issue of bandwidth estimation in a functional nonparametric regression model with function-valued, continuous real-valued and discrete-valued regressors under the framework of unknown error density. Extending from the recent work of Shang (2013) [‘Bayesian Bandwidth Estimation for a Nonparametric Functional Regression Model with Unknown Error Density’, Computational Statistics & Data Analysis, 67, 185–198], we approximate the unknown error density by a kernel density estimator of residuals, where the regression function is estimated by the functional Nadaraya–Watson estimator that admits mixed types of regressors. We derive a likelihood and posterior density for the bandwidth parameters under the kernel-form error density, and put forward a Bayesian bandwidth estimation approach that can simultaneously estimate the bandwidths. Simulation studies demonstrated the estimation accuracy of the regression function and error density for the proposed Bayesian approach. Illustrated by a spectroscopy data set in the food quality control, we applied the proposed Bayesian approach to select the optimal bandwidths in a functional nonparametric regression model with mixed types of regressors.
Asymptotically minimax empirical Bayes estimation of a sparse normal mean vector

For the important classical problem of inference on a sparse high-dimensional normal mean vector, we propose a novel empirical Bayes model that admits a posterior distribution with desirable properties under mild conditions. In particular, our empirical Bayes posterior distribution concentrates on balls, centered at the true mean vector, with squared radius proportional to the minimax rate, and its posterior mean is an asymptotically minimax estimator. We also show that, asymptotically, the support of our empirical Bayes posterior has roughly the same effective dimension as the true sparse mean vector. Simulation from our empirical Bayes posterior is straightforward, and our numerical results demonstrate the quality of our method compared to others having similar large-sample properties.
Topological Consistency via Kernel Estimation

We introduce a consistent estimator for the homology (an algebraic structure representing connected components and cycles) of level sets of both density and regression functions. Our method is based on kernel estimation. We apply this procedure to two problems: (1) inferring the homology structure of manifolds from noisy observations, (2) inferring the persistent homology (a multi-scale extension of homology) of either density or regression functions. We prove consistency for both of these problems. In addition to the theoretical results, we demonstrate these methods on simulated data for binary regression and clustering applications.
On the Connectivity of Fiber Graphs

We consider the connectivity of fiber graphs with respect to Gr\"obner basis and Graver basis moves. First, we present a sequence of fiber graphs using moves from a Gr\"obner basis and prove that their edge-connectivity is lowest possible and can have an arbitrarily large distance from the minimal degree. We then show that graph-theoretic properties of fiber graphs do not depend on the size of the right-hand side. This provides a counterexample to a conjecture of Engstr\"om on the node-connectivity of fiber graphs. Our main result shows that the edge-connectivity in all fiber graphs of this counterexample is best possible if we use moves from Graver basis instead.
Conflict Diagnostics in Directed Acyclic Graphs, with Applications in Bayesian Evidence Synthesis

Complex stochastic models represented by directed acyclic graphs (DAGs) are increasingly employed to synthesise multiple, imperfect and disparate sources of evidence, to estimate quantities that are difficult to measure directly. The various data sources are dependent on shared parameters and hence have the potential to conflict with each other, as well as with the model. In a Bayesian framework, the model consists of three components: the prior distribution, the assumed form of the likelihood and structural assumptions. Any of these components may be incompatible with the observed data. The detection and quantification of such conflict and of data sources that are inconsistent with each other is therefore a crucial component of the model criticism process. We first review Bayesian model criticism, with a focus on conflict detection, before describing a general diagnostic for detecting and quantifying conflict between the evidence in different partitions of a DAG. The diagnostic is a p-value based on splitting the information contributing to inference about a "separator" node or group of nodes into two independent groups and testing whether the two groups result in the same inference about the separator node(s). We illustrate the method with three comprehensive examples: an evidence synthesis to estimate HIV prevalence; an evidence synthesis to estimate influenza case-severity; and a hierarchical growth model for rat weights.
Inference for quantile measures of kurtosis, peakedness, and tail weight

ABSTRACT Insight into measures of peakedness, heavy-tailedness, and kurtosis can be gained by studying Ruppert’s ratios of interquantile ranges. They are not only monotone in Horn’s measure of peakedness when applied to the central portion of the population, but also monotone in the practical tail-index of Morgenthaler and Tukey, when applied to the tails. Non-parametric confidence intervals are found for Ruppert’s ratios, and sample sizes required to obtain such intervals for a pre-specified relative width and level are provided. In addition, the empirical power of distribution-free tests for peakedness and bimodality are found for some symmetric distributions.
Dynamic Principal Components in the Time Domain

We propose a time domain approach to define dynamic principal components (DPC) using a reconstruction of the original series criterion. This approach to define DPC was introduced by Brillinger, who gave a very elegant theoretical solution in the stationary case using the cross spectrum. Our procedure can be applied under more general conditions including the case ofnon stationary series and relatively short series. We also present a robust version of our procedure that allows to estimate the DPC when the series have outlier contamination. Our non robust and robust procedures are illustrated with real datasets.
Isolated effective coherence (iCoh): causal information flow excluding indirect paths

A problem of great interest in real world systems, where multiple time series measurements are available, is the estimation of the intra-system causal relations. For instance, electric cortical signals are used for studying functional connectivity between brain areas, their directionality, the direct or indirect nature of the connections, and the spectral characteristics (e.g. which oscillations are preferentially transmitted). The earliest spectral measure of causality was Akaike's (1968) seminal work on the noise contribution ratio, reflecting direct and indirect connections. Later, a major breakthrough was the partial directed coherence of Baccala and Sameshima (2001) for direct connections. The simple aim of this study consists of two parts: (1) To expose a major problem with the partial directed coherence, where it is shown that it is affected by irrelevant connections to such an extent that it can misrepresent the frequency response, thus defeating the main purpose for which the measure was developed, and (2) To provide a solution to this problem, namely the "isolated effective coherence", which consists of estimating the partial coherence under a multivariate auto-regressive model, followed by setting all irrelevant associations to zero, other than the particular directional association of interest. Simple, realistic, toy examples illustrate the severity of the problem with the partial directed coherence, and the solution achieved by the isolated effective coherence.
Utilizing wind in spatial covariance

This work develops a covariance function which allows for a stronger spatial correlation for pairs of points in the direction of a vector such as wind and weaker for pairs which are perpendicular to it. It derives a simple covariance function by stretching the space along the wind axes (upwind and across wind axes). It is shown that this covariance function is anisotropy in the original space and the functions is explicitly calculated.
Non-probabilistic odds and forecasting with imperfect models

Probability forecasts are intended to account for the uncertainties inherent in forecasting. It is suggested that from an end-user’s point of view probability is not necessarily sufficient to reflect uncertainties that are not simply the result of complexity or randomness, for example, probability forecasts may not adequately account for uncertainties due to model error. It is suggested that an alternative forecast product is to issue nonprobabilistic odds forecasts, which may be as useful to end-users, and give a less distorted account of the uncertainties of a forecast. Our analysis of odds forecasts derives from game theory using the principle that if forecasters truly believe their forecasts, then they should take bets at the odds they offer and not expect to be bankrupted. Despite this game theoretic approach, it is not a market or economic evaluation; it is intended to be a scientific evaluation. Illustrative examples are given of the calculation of odds forecasts and their application to investment, loss mitigation and ensemble weather forecasting.
Understanding Random Forests: From Theory to Practice

Data analysis and machine learning have become an integrative part of the modern scientific methodology, offering automated procedures for the prediction of a phenomenon based on past observations, unraveling underlying patterns in data and providing insights about the problem. Yet, caution should avoid using machine learning as a black-box tool, but rather consider it as a methodology, with a rational thought process that is entirely dependent on the problem under study. In particular, the use of algorithms should ideally require a reasonable understanding of their mechanisms, properties and limitations, in order to better apprehend and interpret their results. Accordingly, the goal of this thesis is to provide an in-depth analysis of random forests, consistently calling into question each and every part of the algorithm, in order to shed new light on its learning capabilities, inner workings and interpretability. The first part of this work studies the induction of decision trees and the construction of ensembles of randomized trees, motivating their design and purpose whenever possible. Our contributions follow with an original complexity analysis of random forests, showing their good computational performance and scalability, along with an in-depth discussion of their implementation details, as contributed within Scikit-Learn. In the second part of this work, we analyse and discuss the interpretability of random forests in the eyes of variable importance measures. The core of our contributions rests in the theoretical characterization of the Mean Decrease of Impurity variable importance measure, from which we prove and derive some of its properties in the case of multiway totally randomized trees and in asymptotic conditions. In consequence of this work, our analysis demonstrates that variable importances [. ].
A biclustering approach to university performances: an Italian case study

University evaluation is a topic of increasing concern in Italy as well as in other countries. In empirical analysis, university activities and performances are often measured by means of indicator variables. The available information are then summarized to respond to different aims. We argue that the evaluation process is a complex phenomenon that cannot be addressed by a simple descriptive approach. In this paper, we used a model-based approach to account for association between indicators and similarities among the observed universities. We examine faculty-level data collected from different sources, covering 55 Italian Economics faculties in the academic year 2009/2010. Making use of a clustering methodology, we introduce a biclustering model that accounts for both homogeneity/heterogeneity among faculties and correlations between indicators. Our results show that there are two substantial different performances between universities which can be strictly related to the nature of the institutions, namely the Private and Public profiles. Each of the two groups has its own peculiar features and its own group-specific list of priorities, strengths and weaknesses. Thus, we suggest that caution should be used in interpreting standard university rankings as they generally do not account for the complex structure of the data.
Lognormal Distributions and Geometric Averages of Positive Definite Matrices

This article gives a formal definition of a lognormal family of probability distributions on the set of symmetric positive definite (PD) matrices, seen as a matrix-variate extension of the univariate lognormal family of distributions. Two forms of this distribution are obtained as the large sample limiting distribution via the central limit theorem of two types of geometric averages of i.i. d. PD matrices: the log-Euclidean average and the canonical geometric average. These averages correspond to two different geometries imposed on the set of PD matrices. The limiting distributions of these averages are used to provide large-sample confidence regions for the corresponding population means. The methods are illustrated on a voxelwise analysis of diffusion tensor imaging data, permitting a comparison between the various average types from the point of view of their sampling variability.
Fully Adaptive Density-Based Clustering

The clusters of a distribution are often defined by the connected components of a density level set. However, this definition depends on the user-specified level. We address this issue by proposing a simple, generic algorithm, which uses an almost arbitrary level set estimator to estimate the smallest level at which there are more than one connected components. In the case where this algorithm is fed with histogram-based level set estimates, we provide a finite sample analysis, which is then used to show that the algorithm consistently estimates both the smallest level and the corresponding connected components. We further establish rates of convergence for the two estimation problems, and last but not least, we present a simple, yet adaptive strategy for determining the width-parameter of the involved density estimator in a data-depending way.
Cosmic structure and dynamics of the local universe

We present a cosmography analysis of the local Universe based on the recently released Two-Micron All-Sky Redshift Survey catalogue. Our method is based on a Bayesian Networks Machine Learning algorithm (the Kigen-code) which self-consistently samples the initial density fluctuations compatible with the observed galaxy distribution and a structure formation model given by second-order Lagrangian perturbation theory (2LPT). From the initial conditions we obtain an ensemble of reconstructed density and peculiar velocity fields which characterize the local cosmic structure with high accuracy unveiling non-linear structures like filaments and voids in detail. Coherent redshift-space distortions are consistently corrected within 2LPT. From the ensemble of cross-correlations between the reconstructions and the galaxy field and the variance of the recovered density fields, we find that our method is extremely accurate up to k∼ 1 h Mpc−1 and still yields reliable results down to scales of about 3–4 h−1 Mpc. The motion of the Local Group we obtain within ∼80 h−1 Mpc (vLG = 522 ± 86 km s−1, lLG = 291° ± 16°, bLG = 34° ± 8°) is in good agreement with measurements derived from the cosmic microwave background and from direct observations of peculiar motions and is consistent with the predictions of ΛCDM.
A modified $χ^2$-test for uplift models with applications in marketing performance measurement

Uplift, essentially being the difference between two probabilities, is a central number in marketing performance measurement. A frequent question in applications is whether the uplifts of two campaigns are significantly different. In this article we present a new $\chi^2$-statistic which allows to answer this question by performing a statistical test. We show that this statistic is asymptotically $\chi^2$-distributed and demonstrate its application in a real life example. By running simulations with this new and alternative approaches, we find our suggested test to exhibit a better decisive power.
Particle Filtering and Smoothing Using Windowed Rejection Sampling

"Particle methods" are sequential Monte Carlo algorithms, typically involving importance sampling, that are used to estimate and sample from joint and marginal densities from a collection of a, presumably increasing, number of random variables. In particular, a particle filter aims to estimate the current state $X_{n}$ of a stochastic system that is not directly observable by estimating a posterior distribution $\pi(x_{n}|y_{1},y_{2}, \ldots, y_{n})$ where the $\{Y_{n}\}$ are observations related to the $\{X_{n}\}$ through some measurement model $\pi(y_{n}|x_{n})$. A particle smoother aims to estimate a marginal distribution $\pi(x_{i}|y_{1},y_{2}, \ldots, y_{n})$ for $1 \leq i < n$. Particle methods are used extensively for hidden Markov models where $\{X_{n}\}$ is a Markov chain as well as for more general state space models. Existing particle filtering algorithms are extremely fast and easy to implement. Although they suffer from issues of degeneracy and "sample impoverishment", steps can be taken to minimize these problems and overall they are excellent tools for inference. However, if one wishes to sample from a posterior distribution of interest, a particle filter is only able to produce dependent draws. Particle smoothing algorithms are complicated and far less robust, often requiring cumbersome post-processing, "forward-backward" recursions, and multiple passes through subroutines. In this paper we introduce an alternative algorithm for both filtering and smoothing that is based on rejection sampling "in windows" . We compare both speed and accuracy of the traditional particle filter and this "windowed rejection sampler" (WRS) for several examples and show that good estimates for smoothing distributions are obtained at no extra cost.