Miracle-dz/UniFAR-FTUs huggingface dataset:
Discrete perceptrons

Perceptrons have been known for a long time as a promising tool within the neural networks theory. The analytical treatment for a special class of perceptrons started in seminal work of Gardner [11]. Techniques initially employed to characterize perceptrons relied on a statistical mechanics approach. Many of such predictions obtained in [11] (and in a follow-up [12]) were later on established rigorously as mathematical facts (see, e.g. [22–24,29,31,34]). These typically related to spherical perceptrons. A lot of work has been done related to various other types of perceptrons. Among the most challenging ones are what we will refer to as the discrete perceptrons. An introductory statistical mechanics treatment of such perceptrons was given in [15]. Relying on results of [11], [15] characterized many of the features of several types of discrete perceptrons. We in this paper, consider a similar subclass of discrete perceptrons and provide a mathematically rigorous set of results related to their performance. As it will turn out, many of the statistical mechanics predictions obtained for discrete predictions will in fact appear as mathematically provable bounds. This will in a way emulate a similar type of behavior we observed in [24,29,31] when studying spherical perceptrons.
ESTIMATION OF TREND IN STATE-SPACE MODELS: ASYMPTOTIC MEAN SQUARE ERROR AND RATE OF CONVERGENCE

The focus of this paper is on trend estimation for a general state-space model Y t = μ t + e t , where the dth difference of the trend {μ t } is assumed to be i.i. d. , and the error sequence {e t is assumed to be a mean zero stationary process. A fairly precise asymptotic expression of the mean square error is derived for the estimator obtained by penalizing the dth order differences. Optimal rate of convergence is obtained, and it is shown to be "asymptotically equivalent" to a nonparametric estimator of a fixed trend model of smoothness of order d — 0.5. The results of this paper show that the optimal rate of convergence for the stochastic and nonstochastic cases are different. A criterion for selecting the penalty parameter and degree of difference d is given, along with an application to the global temperature data, which shows that a longer term history has nonlinearities that are important to take into consideration.
Weak limits for exploratory plots in the analysis of extremes

Exploratory data analysis is often used to test the goodness-of-fit of sample observations to specific target distributions. A few such graphical tools have been extensively used to detect subexponential or heavy-tailed behavior in observed data. In this paper we discuss asymptotic limit behavior of two such plotting tools: the quantile-quantile plot and the mean excess plot. The weak consistency of these plots to fixed limit sets in an appropriate topology of $\mathbb{R}^2$ has been shown in Das and Resnick (Stoch. Models 24 (2008) 103-132) and Ghosh and Resnick (Stochastic Process. Appl. 120 (2010) 1492-1517). In this paper we find asymptotic distributional limits for these plots when the underlying distributions have regularly varying right-tails. As an application we construct confidence bounds around the plots which enable us to statistically test whether the underlying distribution is heavy-tailed or not.
Bayesian analysis of measurement error models using INLA

To account for measurement error (ME) in explanatory variables, Bayesian approaches provide a flexible framework, as expert knowledge about unobserved covariates can be incorporated in the prior distributions. However, given the analytic intractability of the posterior distribution, model inference so far has to be performed via time-consuming and complex Markov chain Monte Carlo implementations. In this paper we extend the Integrated nested Laplace approximations (INLA) approach to formulate Gaussian ME models in generalized linear mixed models. We present three applications, and show how parameter estimates are obtained for common ME models, such as the classical and Berkson error model including heteroscedastic variances. To illustrate the practical feasibility, R-code is provided.
Assessing Relative Volatility/Intermittency/Energy Dissipation

We introduce the notion of relative volatility/intermittency and demonstrate how relative volatility statistics can be used to estimate consistently the temporal variation of volatility/intermittency even when the data of interest are generated by a non-semimartingale, or a Brownian semistationary process in particular. While this estimation method is motivated by the assessment of relative energy dissipation in empirical data of turbulence, we apply it also to energy price data. Moreover, we develop a probabilistic asymptotic theory for relative power variations of Brownian semistationary processes and Ito semimartingales and discuss how it can be used for inference on relative volatility/intermittency.
n-DIGIT BENFORD DISTRIBUTED RANDOM VARIABLES

AZAR KHOSRAVANI AND CONSTANTIN RASINARIUAbstract. The scope of this paper is twofold. First, to emphasize the use of the mod 1map in exploring the digit distribution of random variables. We show that the well-knownbase- and scale-invariance of Benford variables are consequences of their associated mod1 density functions being uniformly distributed. Second, to introduce a new concept ofthe n-digit Benford variable. Such a variable is Benford in the ﬁrst ndigits, but it is notguaranteed to have a logarithmic distribution beyond the n-th digit. We conclude thepaper by giving a general construction method for n-digit Benford variables, and providea concrete example.
Bayesian Decision-optimal Interval Designs for Phase I Clinical Trials

Interval designs are a class of phase I trial designs for which the decision of dose assignment is determined by comparing the observed toxicity rate at the current dose with a prespecified (toxicity tolerance) interval. If the observed toxicity rate is located within the interval, we retain the current dose; if the observed toxicity rate is greater than the upper boundary of the interval, we deescalate the dose; and if the observed toxicity rate is smaller than the lower boundary of the interval, we escalate the dose. The most critical issue for the interval design is choosing an appropriate interval so that the design has good operating characteristics. By casting dose finding as a Bayesian decision-making problem, we propose new flexible methods to select the interval boundaries so as to minimize the probability of inappropriate dose assignment for patients. We show, both theoretically and numerically, that the resulting optimal interval designs not only have desirable finite- and large-sample properties, but also are particularly easy to implement in practice. Compared to existing designs, the proposed (local) optimal design has comparable average performance, but a lower risk of yielding a poorly performing clinical trial.
GARCH-extended models: theoretical properties and applications

This paper is concerned with some properties of the generalized GARCH models, obtained by extending GARCH models with exogenous variables, the so-called GARCH extended (GARCHX) models. For these, we establish sufficient conditions for some properties such as stationarity, existence of moments, ergodicity, geometric ergodicity, consistence and asymptotic normality of likelihood estimators of the model parameters. For some of these properties we show that the conditions that we propose are also necessary. We further provide examples and applications to illustrate and highlight the importance of our findings.
Statistical Inferences Using Large Estimated Covariances for Panel Data and Factor Models

While most of the convergence results in the literature on high dimensional covariance matrix are concerned about the accuracy of estimating the covariance matrix (and precision matrix), relatively less is known about the effect of estimating large covariances on statistical inferences. We study two important models: factor analysis and panel data model with interactive effects, and focus on the statistical inference and estimation efficiency of structural parameters based on large covariance estimators. For efficient estimation, both models call for a weighted principle components (WPC), which relies on a high dimensional weight matrix. This paper derives an efficient and feasible WPC using the covariance matrix estimator of Fan et al. (2013). However, we demonstrate that existing results on large covariance estimation based on absolute convergence are not suitable for statistical inferences of the structural parameters. What is needed is some weighted consistency and the associated rate of convergence, which are obtained in this paper. Finally, the proposed method is applied to the US divorce rate data. We find that the efficient WPC identifies the significant effects of divorce-law reforms on the divorce rate, and it provides more accurate estimation and tighter confidence intervals than existing methods.
Efficient Bayesian inference for long memory processes

In forecasting problems it is important to know whether or not recent events represent a regime change (low long-term predictive potential), or rather a local manifestation of longer term effects (potentially higher predictive potential). Mathematically, a key question is about whether the underlying stochastic process exhibits "memory", and if so whether the memory is "long" in a precise sense. Being able to detect or rule out such effects can have a profound impact on speculative investment (e.g., in financial markets) and inform public policy (e.g., characterising the size and timescales of the earth system's response to the anthropogenic CO2 perturbation). Most previous work on inference of long memory effects is frequentist in nature. Here we provide a systematic treatment of Bayesian inference for long memory processes via the Autoregressive Fractional Integrated Moving Average (ARFIMA) model. In particular, we provide a new approximate likelihood for efficient parameter inference, and show how nuisance parameters (e.g., short memory effects) can be integrated over in order to focus on long memory parameters and hypothesis testing more directly than ever before. We illustrate our new methodology on both synthetic and observational data, with favorable comparison to the standard estimators.
A Dynamic Bi-orthogonal Field Equation Approach for Efficient Bayesian Calibration of Large-Scale Systems

This paper proposes a novel computationally efficient dynamic bi-orthogonality based approach for calibration of a computer simulator with high dimensional parametric and model structure uncertainty. The proposed method is based on a decomposition of the solution into mean and a random field using a generic Karhunnen-Loeve expansion. The random field is represented as a convolution of separable Hilbert spaces in stochastic and spacial dimensions that are spectrally represented using respective orthogonal bases. In particular, the present paper investigates generalized polynomial chaos bases for stochastic dimension and eigenfunction bases for spacial dimension. Dynamic orthogonality is used to derive closed form equations for the time evolution of mean, spacial and the stochastic fields. The resultant system of equations consists of a partial differential equation (PDE) that define dynamic evolution of the mean, a set of PDEs to define the time evolution of eigenfunction bases, while a set of ordinary differential equations (ODEs) define dynamics of the stochastic field. This system of dynamic evolution equations efficiently propagates the prior parametric uncertainty to the system response. The resulting bi-orthogonal expansion of the system response is used to reformulate the Bayesian inference for efficient exploration of the posterior distribution. Efficacy of the proposed method is investigated for calibration of a 2D transient diffusion simulator with uncertain source location and diffusivity. Computational efficiency of the method is demonstrated against a Monte Carlo method and a generalized polynomial chaos approach.
Efficient Metropolis–Hastings Proposal Mechanisms for Bayesian Regression Tree Models

Bayesian regression trees are flexible non-parametric models that are well suited to many modern statistical regression problems. Many such tree models have been proposed, from the simple single- tree model to more complex tree ensembles. Their non-parametric formulation allows for effective and efficient modeling of datasets exhibiting complex non-linear relationships between the model pre- dictors and observations. However, the mixing behavior of the Markov Chain Monte Carlo (MCMC) sampler is sometimes poor. This is because the proposals in the sampler are typically local alterations of the tree structure, such as the birth/death of leaf nodes, which does not allow for efficient traversal of the model space. This poor mixing can lead to inferential problems, such as under-representing uncertainty. In this paper, we develop novel proposal mechanisms for efficient sampling. The first is a rule perturbation proposal while the second we call tree rotation. The perturbation proposal can be seen as an efficient variation of the change proposal found in existing literature. The novel tree rotation proposal is simple to implement as it only requires local changes to the regression tree structure, yet it efficiently traverses disparate regions of the model space along contours of equal probability. When combined with the classical birth/death proposal, the resulting MCMC sampler exhibits good acceptance rates and properly represents model uncertainty in the posterior samples. We implement this sampling algorithm in the Bayesian Additive Regression Tree (BART) model and demonstrate its effectiveness on a prediction problem from computer experiments and a test function where structural tree variability is needed to fully explore the posterior.
Sparse Principal Component Analysis: a Least Squares approximation approach

Sparse Principal Components Analysis aims to find principal components with few non-zero loadings. We derive such sparse solutions by adding a genuine sparsity requirement to the original Principal Components Analysis (PCA) objective function. This approach differs from others because it preserves PCA's original optimality: \uns\ of the components and Least Squares approximation of the data. To identify the best subset of non-zero loadings we propose a Branch-and-Bound search and an iterative elimination algorithm. This last algorithm finds sparse solutions with large loadings and can be run without specifying the cardinality of the loadings and the number of components to compute in advance. We give thorough comparisons with the existing Sparse PCA methods and several examples on real datasets.
Iterative Scaling in Curved Exponential Families

The paper describes a generalized iterative proportional fitting procedure that can be used for maximum likelihood estimation in a special class of the general log‐linear model. The models in this class, called relational, apply to multivariate discrete sample spaces that do not necessarily have a Cartesian product structure and may not contain an overall effect. When applied to the cell probabilities, the models without the overall effect are curved exponential families and the values of the sufficient statistics are reproduced by the MLE only up to a constant of proportionality. The paper shows that Iterative Proportional Fitting, Generalized Iterative Scaling, and Improved Iterative Scaling fail to work for such models. The algorithm proposed here is based on iterated Bregman projections. As a by‐product, estimates of the multiplicative parameters are also obtained. An implementation of the algorithm is available as an R‐package.
Bayesian protein sequence and structure alignment

The structure of a protein is crucial in determining its functionality and is much more conserved than sequence during evolution. A key task in structural biology is to compare protein structures to determine evolutionary relationships, to estimate the function of newly discovered structures and to predict unknown structures. We propose a Bayesian method for protein structure alignment, with the prior on alignments based on functions which penalize ‘gaps’ in the aligned sequences. We show how a broad class of penalty functions fits into this framework, and how the resulting posterior distribution can be efficiently sampled. A commonly used gap penalty function is shown to be a special case, and we propose a new penalty function which alleviates an undesirable feature of the commonly used penalty. We illustrate our method on benchmark data sets and find that it competes well with popular tools from computational biology. Our method has the benefit of being able potentially to explore multiple competing alignments and to quantify their merits probabilistically. The framework naturally enables further information such as amino acid sequence to be included and could be adapted to other situations such as flexible proteins or domain swaps.
Bayesian nonparametric inference on the Stiefel manifold

The Stiefel manifold $V_{p,d}$ is the space of all $d \times p$ orthonormal matrices, with the $d-1$ hypersphere and the space of all orthogonal matrices constituting special cases. In modeling data lying on the Stiefel manifold, parametric distributions such as the matrix Langevin distribution are often used; however, model misspecification is a concern and it is desirable to have nonparametric alternatives. Current nonparametric methods are Frechet mean based. We take a fully generative nonparametric approach, which relies on mixing parametric kernels such as the matrix Langevin. The proposed kernel mixtures can approximate a large class of distributions on the Stiefel manifold, and we develop theory showing posterior consistency. While there exists work developing general posterior consistency results, extending these results to this particular manifold requires substantial new theory. Posterior inference is illustrated on a real-world dataset of near-Earth objects.
The tomb next door: An update to ``Statistical analysis of an archeological find''

In June of 2010 access via robotic means was obtained to a tomb adjacent to the one studied in Feuerverger [Ann. Appl. Stat. 2 (2008) 3-54]. In this update, we lay out and attempt to interpret the remarkable findings from this second tomb and comment on the statistical and scientific significance of these new data and of their possible inferential connections to the data from the first tomb. Readers are then invited to formulate their own conclusions.
A new perspective on least squares under convex constraint

Consider the problem of estimating the mean of a Gaussian random vector when the mean vector is assumed to be in a given convex set. The most natural solution is to take the Euclidean projection of the data vector on to this convex set; in other words, performing "least squares under a convex constraint. " Many problems in modern statistics and statistical signal processing theory are special cases of this general situation. Examples include the lasso and other high-dimensional regression techniques, function estimation problems, matrix estimation and completion, shape-restricted regression, constrained denoising, linear inverse problems, etc. This paper presents three general results about this problem, namely, (a) an exact computation of the main term in the estimation error by relating it to expected maxima of Gaussian processes (existing results only give upper bounds), (b) a theorem showing that the least squares estimator is always admissible up to a universal constant in any problem of the above kind and (c) a counterexample showing that least squares estimator may not always be minimax rate-optimal. The result from part (a) is then used to compute the error of the least squares estimator in two examples of contemporary interest.
A flexible bivariate location-scale finite mixture approach to economic growth

We introduce a multivariate multidimensional mixed-effects regression model in a finite mixture framework. We relax the usual unidimensionality assumption on the random effects multivariate distribution. Thus, we introduce a multidimensional multivariate discrete distribution for the random terms, with a possibly different number of support points in each univariate profile, allowing for a full association structure. Our approach is motivated by the analysis of economic growth. Accordingly, we define an extended version of the augmented Solow model. Indeed, we allow all model parameters, and not only the mean, to vary according to a regression model. Moreover, we argue that countries do not follow the same growth process, and that a mixture-based approach can provide a natural framework for the detection of similar growth patterns. Our empirical findings provide evidence of heterogenous behaviors and suggest the need of a flexible approach to properly reflect the heterogeneity in the data. We further test the behavior of the proposed approach via a simulation study, considering several factors such as the number of observed units, times and levels of heterogeneity in the data.
A likelihood-based scoring method for peptide identification using mass spectrometry

Mass spectrometry provides a high-throughput approach to identify proteins in biological samples. A key step in the analysis of mass spectrometry data is to identify the peptide sequence that, most probably, gave rise to each observed spectrum. This is often tackled using a database search: each observed spectrum is compared against a large number of theoretical "expected" spectra predicted from candidate peptide sequences in a database, and the best match is identified using some heuristic scoring criterion. Here we provide a more principled, likelihood-based, scoring criterion for this problem. Specifically, we introduce a probabilistic model that allows one to assess, for each theoretical spectrum, the probability that it would produce the observed spectrum. This probabilistic model takes account of peak locations and intensities, in both observed and theoretical spectra, which enables incorporation of detailed knowledge of chemical plausibility in peptide identification. Besides placing peptide scoring on a sounder theoretical footing, the likelihood-based score also has important practical benefits: it provides natural measures for assessing the uncertainty of each identification, and in comparisons on benchmark data it produced more accurate peptide identifications than other methods, including SEQUEST. Although we focus here on peptide identification, our scoring rule could easily be integrated into any downstream analyses that require peptide-spectrum match scores.
Role of normalization in spectral clustering for stochastic blockmodels

Spectral clustering is a technique that clusters elements using the top few eigenvectors of their (possibly normalized) similarity matrix. The quality of spectral clustering is closely tied to the convergence properties of these principal eigenvectors. This rate of convergence has been shown to be identical for both the normalized and unnormalized variants in recent random matrix theory literature. However, normalization for spectral clustering is commonly believed to be beneficial [Stat. Comput. 17 (2007) 395-416]. Indeed, our experiments show that normalization improves prediction accuracy. In this paper, for the popular stochastic blockmodel, we theoretically show that normalization shrinks the spread of points in a class by a constant fraction under a broad parameter regime. As a byproduct of our work, we also obtain sharp deviation bounds of empirical principal eigenvalues of graphs generated from a stochastic blockmodel.