Query-Driven Sampling for Collective Entity Resolution September 1 , 2018

Probabilistic databases play a preeminent role in the processing and management of uncertain data. Recently, many database research efforts have integrated probabilistic models into databases to support tasks such as information extraction and labeling. Many of these efforts are based on batch oriented inference which inhibits a realtime workflow. One important task is entity resolution (ER). ER is the process of determining records (mentions) in a database that correspond to the same real-world entity. Traditional pairwise ER methods can lead to inconsistencies and low accuracy due to localized decisions. Leading ER systems solve this problem by collectively resolving all records using a probabilistic graphical model and Markov chain Monte Carlo (MCMC) inference. However, for large datasets this is an extremely expensive process. One key observation is that, such exhaustive ER process incurs a huge up-front cost, which is wasteful in practice because most users are interested in only a small subset of entities. In this chapter, we advocate pay-as-you-go entity resolution by developing a number of query-driven collective ER techniques. We introduce two classes of SQL queries that involve ER operators — selectiondriven ER and join-driven ER. We implement novel variations of the MCMC Metropolis Hastings algorithm to generate biased samples and selectivity-based scheduling algorithms to support the two classes of ER queries. Finally, we show that query-driven ER algorithms can converge and return results within minutes over a database populated with the extraction from a newswire dataset containing 71 million mentions. 1 Query-Driven Entity Resolution Introduction Entity resolution (ER) is the process of identifying and linking/grouping different manifestations (e.g., mentions, noun phrases, named entities) of the same real world object. It is a crucial task for many applications including knowledge base construction, information extraction, and question answering. For decades, ER has been studied in both database and natural language processing communities to link database records or to perform entity resolution over extracted mentions (noun phrases) in text. ER is a notoriously difficult and expensive task. Traditionally, entities are resolved using strict pairwise similarity, which usually leads to inconsistencies and low accuracy due to localized, myopic decisions [39]. More recently, collective entity resolution methods have achieved state-of-the-art accuracy because they leverage relational information in the data to determine resolution jointly rather than independently [5]. However, it is expensive to run collective ER based on probabilistic graphical models (GMs), especially for cross-document entity resolution, where ER must be performed over millions of mentions. In many previous approaches, collective ER is performed exhaustively over all the mentions in a data set, returning all entities. Researchers have developed new methods to perform large-scale cross-document entity resolution over parallel frameworks [33, 39]. However, in many ER applications, users are only interested in one or a small subset of entities. This key observation motivates query-driven ER, an alternative approach to solving the scalability problem for ER. Compared to previous ER models and algorithms, query-driven techniques in this chapter scale to data sets that are in many cases three orders of magnitude larger. Moreover, the ER model in this chapter is general enough to take both bibliographic records and mentions extracted from unstructured text. Query-driven ER techniques over GMs can also be generalized for other applications to perform query-driven inference. This work follows a line of research on implementing ML models inside of databases [18, 22,38]. Researchers use factor graphs because this flexible representation works well with other machine learning algorithms. ER is ubiquitous and an important part of many analytic pipelines; a probabilistic database implementation is natural. In this chapter, we first introduce SQL-like queries that involve ER operations. These ER operators are an SQL comparison operator (i.e., ER-based equality) that returns true if two mentions map to the same entity. ar X iv :1 50 8. 03 11 6v 1 [ cs . D B ] 1 3 A ug 2 01 5 Factor Graphs, a type of GM, are used to model the collective entity resolution over extracted mentions from text. Using this ER based comparison operator, users can pose selection queries to find all mentions that map to a single entity or pose join queries to find mentions that map to the subset of entities that they are interested in resolving. Because exhaustive ER is expensive it is common to use blocking techniques to partition the data set into approximately similar groups called canopies. Query-driven ER in this chapter differs from blocking in two important ways: 1) deterministic blocks are replaced by a pairwise distance-based metric, and 2) blocks (or canopies) are implicit to the query-driven ER data set and do not have to be created in advanced. The latter point, implicit blocking, is realized using a data structure created based on the similarity to a query mention. This data structure allows parameters to include or remove mentions from the working data set. This property is similar to the iterative blocking technique [37], which is shown to improve ER accuracy. Such an approach can dramatically amortize the overall ER cost suitable for the pay-as-you-go paradigm in dataspaces [25]. To support ER driven by queries, we develop three sampling algorithms for MCMC inference over graphical models. More specifically, instead of a uniform sampling distribution, we sample on a distribution that is biased to the query. We develop a query-driven sampling techniques that maximizes the resolution of the target query entity (target-fixed) and biases the samples based on the pairwise similarity metric between mentions and query nodes (query-proportional). We also introduce a hybrid method that performs query-proportional sampling over a fixed target. We develop two optimizations to the query-proportional and hybrid methods to model the similarity and dissimilarity between the mentions and the query entity, i.e., attract and repel scores. In the first target-fixed algorithm, we adapt the samples to resolve the query entity. The second query-proportional algorithm, selects mentions based on their probabilistic similarity to the query entity. The third hybrid algorithm combines the two approaches. A summary of approaches can be found in Table 3. When a user is interested in resolving more than one entity we employ multi-node ER techniques. To implement multi-node ER queries, single-node ER techniques may be naively performed iteratively to resolve one entity at a time. However, such an algorithm can lead to un-optimized resource allocation if the same number of samples is generated for each target entity, or low throughput if one of the entities has a disproportionately low convergence rate. To alleviate this problem, we present three multi-query ER algorithms that schedule the sample generation among query nodes in order to improve overall convergence rate. In summary, the contributions of this chapter are the following: • We define a query-driven ER problem for cross-document, collective ER over text extracted from unstructured data sets; • We develop three single-node algorithms that perform focused sampling and reduce convergence time by orders-of-magnitude compared to a non-query-driven baseline (Section 4). We develop two influence functions that use attract and repel techniques to grow or shrink query entities (Section 5.1); • We develop scheduling algorithms to optimize the overall convergence rate of the multi-query ER (Section 5.2). The best scheduling algorithm is based on selectivity of different target entities (Section 5.3). The results show that query-driven ER algorithms is a promising method of enabling realtime, ad-hoc, ERbased queries over large data sets. Single node queries of different selectivity converge to a high-quality entity within 1-2 minutes over a newswire data set containing 71 million mentions. Experiments also show that such real-time ER query answering allows users to iteratively refine ER queries by adding context to achieve better accuracy (Section 6). 2 Query-Driven Entity Resolution Preliminaries In this section we present a foundation of concepts discussed in this chapter. We start with an introduction of factor graphs then discuss sampling techniques over this model. Finally, we formally introduce state-of-the-art entity resolution approaches and explain the origin. 2.1 Factor Graphs Graphical models are a formalism for specifying complex probability distributions over many interdependent random variables. Factor graphs are bipartite graphical models that can capture arbitrary relationships between random variables through the use of factors [21]. As depicted in Figure 1, links always connect random variables (represented as circles) and factor nodes (represented as black squares). Factors are Figure 1: Three node factor graph. Circles (random variables) with mi represent mentions and those with ei represent entities. Clouds are added for visual emphasis of entity clusters functions that take as input the current setting of connected random variables, and output a positive real-valued scalar indicating the compatibility of the random variables settings. The probability of a setting to all the random variables is a normalized product of all the factors. Intuitively, the highest probability settings have variable assignments that yield the highest factor scores. We use factor graphs to represent complex entity resolution relationships. Nodes (random variables) may correspond to mentions of people, places and organizations in documents. Nodes also represent the random variables that correspond to groups of mentions (entities), these nodes