Graphical models beyond standard settings: lifted decimation, labeling, and counting

The influence of Artificial Intelligence (AI) and Machine Learning on our everyday lives has been growing constantly over the past decades. Applications in both areas have attracted much attention and several of them depend on Probabilistic Graphical Models (PGMs). Furthermore, the emerging interest in methods originating from the Statistical Relational Learning (SRL) community has added to this underlying trend over the past ten years. In many cases, the combination of logic and probability is superior to relying on only either of the two. However, with problems becoming larger and more complex, inference and learning are still major issues in PGMs and in particular in the context of models derived from SRL formalisms. On the other hand, many problems in SRL are specified in such a way that symmetries arise from the underlying model structure. Exploiting these symmetries during inference, which is referred to as “lifted inference”, has lead to significant efficiency gains. The idea of lifted inference is not solely applicable to relational probabilistic models though. We provide several enhanced versions of known algorithms that show to be liftable too and thereby apply lifting in “non-standard” settings. For example, we show how to use lifted message passing in a decimation procedure for solving satisfying problems. By doing so, we extend the understanding of the applicability of lifted inference and lifting in general. The majority of research on lifted inference has so far been concentrating on PGMs with binary random variables. Certainly, these models are not the universal remedy to all problems in AI and in fact represent only a small subset of all PGMs. In many cases, we wish to model variables over a large domain, or even infinite range, even if this introduces new challenges to the complexity. Often, common formalisms become unsuitable and even efficient approximative inference is intractable. However, we can still transfer ideas from SRL and lifting to other approaches that are more suitable in this context. In this regard, we show that algorithms solving multiclass labeling problems can benefit from SRL-like weighted rules and lifted inference — which are both non-standard techniques in this area. Here, weighted rules induce sparse and symmetric problems from which Label Propagation, an effective labeling algorithm, can benefit. We use our novel Label Propagation approach in combination with an innovative Web-based data harvesting pipeline to label author-paper-pairs with geographic information in online bibliographies. This results is a large-scale transnational bibliography containing affiliation information over time for roughly one million authors. By counting the number of migration hops of researchers, we can infer mobility patterns in migration from this database. We find distributions fitting the data well which have also been found describing human and social phenomena in other areas. In this line of research, we also show that the Web provides a rich set of additional data sources where similar patterns can be found in the spread of information and content on the Web. These findings share one commonality: all datasets were build upon counting. Although counting is done literally everywhere, mainstream PGMs have widely been neglecting count data. In the case where the ranges of the random variables are defined over the natural numbers, crude approximations to the true distribution are often made by discretization or a Gaussian assumption. To handle count data with positive and negative dependencies, we introduce Poisson Dependency Networks (PDNs). PDNs are defined over a set of local probability distributions and we obtain a joint distribution via sampling, allowing us to answer various probabilistic queries. Additionally, PDNs do not only allow to describe dependencies between count random variables but also allow to make predictions for unseen instances. Hence, we provide a new class of non-standard PGMs naturally handling count data.