Bayesian nonparametric methods for non-exchangeable random structures

Bayesian Statistics has been increasingly popular in the last five decades. Besides having decision theoretic foundation, the Bayesian approach found popularity thanks to the simple and intuitive framework that it offers to model phenomena, allowing the statistician to include prior knowledge in the inference procedure. Each unknown quantity in the model is endowed with a prior distribution which encodes our prior belief, and Bayes' rule functions as a probabilistic tool to update the prior belief after observing the data. The result of the inference scheme is the posterior distribution, which, being a probability distribution on the unknown parameters, automatically provides a principled way to quantify the uncertainty of our estimation and make prediction. The choice of the prior distribution can be crucial for the performance of the Bayesian model. In particular, restricting the prior distribution to a finite-dimensional space can lead to a model which may not fit the data properly. Very frequently in applications, in order to capture complex patterns of the data, statistical models include infinite-dimensional parameters, and in such cases the model is referred to as nonparametric.
The first part of this thesis is dedicated to Bayesian Nonparametric modelling. In particular the focus will be on modelling two discrete random structures: random partitions and random feature allocations. These two objects are the building blocks for several models used for clustering, feature modeling, network modeling, regression, density estimation. They allow to describe dataset of increasing complexity, such as collections of labelled observations with an increasing number of labels, and observations showing features with the number of features growing over time. A recurrent assumption in Bayesian modelling is the exchangeability of the data. Despite being a reasonable and convenient assumption, it leads to strict asymptotic behaviour of the model. In clustering application for instance, exchangeability in random partition models implies that the size of each cluster grows linearly with the number of observations. Analogously, in feature modelling, exchangeable feature allocation models have the number of objects sharing the same feature grow linearly with the number of objects observed. Such restrictive asymptotic properties are intrinsic in exchangeable models and are not appropriate for some applications, therefore it is necessary to go beyond exchangeability in order to design priors with a broader range of asymptotic properties. Moreover, we show that the proposed priors retain good properties already achieved by exchangeable models, such as the power-law property, which is ubiquitous in real-world applications.
On the one hand Bayesian modelling gives freedom by allowing to choose the prior distribution which best reflects our prior belief. On the other hand, especially in nonparametric settings, the design of such priors could be challenging and one would rather opt for standard and objective priors to use in practice. One way to validate and compare priors consists in a frequentist analysis of the posterior distribution. Consistency of the posterior distribution for example, ensures that the effect of the prior knowledge vanishes as an increasing amount of data is collected, while posterior contraction rates can be used to compare different priors. In the second part of the thesis we study asymptotic properties for different priors in Functional Data Analysis. The latter has gained an increasing attention in the last decades thanks to the raise of data collected over fine grids, making it reasonable to model them as functions. While Functional Data Analysis has been thoroughly studied in the frequentist literature and widely used in Bayesian modelling, asymptotic analysis of Bayesian methods are rather scarce. We will focus on two models: Functional Linear Regression and Functional Single-Index Model, both in the setting of functional predictor and scalar response. In particular we will show that for two classes of priors used for Bayesian Functional Linear Regression, minimax posterior contraction rates can be achieved. Moreover, posterior contraction rates are derived for a class of priors for the Bayesian Functional Single-Index Models.

Keywords: clustering
Bayesian nonparametrics
Bayesian asymptotics
exchangeability
Advisor: Caron, F
Teh, Y
Rousseau, J