Entities on the Web - Resolution, Matching and Profiling

The majority of the information on the web is encoded as web documents in natural language for human consumption. According to International Data Corporation (IDC) 80% of the data on the web is unstructured (free text) and is growing at a rapid pace due to the ease with which data can be published on the blogs, social networks, web, etc. The fundamental idea of Semantic Web is to link all the knowledge on the web. For Semantic Web to be widely adopted, and to exploit its full potential, it is important that the researched techniques understand and automatically extract knowledge from the unstructured web documents, as majority of data on the web is unstructured. A promising approach to have programmatic access to such knowledge is the use of information extraction techniques. Most frequently these techniques aim at extracting entities, such as persons, geographic locations, etc., from free text. These entities can potentially be linked to each other, thus creating a de-facto global knowledge graph of linked entities. A number of entity-related challenges need to be addressed for realizing the entity-oriented view of Semantic Web. In this doctoral thesis, we provide research contributions to the field of entity extraction from Web text documents with the aim of facilitating the adoption of the Semantic Web. This thesis addresses following entity-related problems: Entity Resolution for web documents; Entity Matching in microblogging environments; and Entity Profiling and Applications. More specifically, we make the following contributions: (1) Entity Resolution for Web Documents: One of the key challenges to realize automated processing of the information on the Web is related to the entity resolution problem. There are a number of tools that reliably recognize named entities, such as persons, companies, geographic locations, in Web documents. The names of these extracted entities are however non-unique; the same name on different Web pages might or might not refer to the same entity. We address this disambiguation problem, which is very similar to the entity resolution problem studied in relational databases, however there are also several differences. Most importantly Web pages often only contain partial or incomplete information about the entities. We propose a generic framework where multiple similarity functions corresponding to the domain specific rules can be defined. We make use of techniques from graph theory and machine learning for efficiently combining the evidence from multiple similarity functions for improved ER results, and demonstrate the efficiency of our framework on two real-world datasets. (2) Entities in micro-blogs like Twitter: Twitter is a popular micro-blogging service on the Web, where people can publish short messages, which then become visible to other users of the service. While the topics of these messages vary, there are a lot of messages where the users express their opinions about companies or their products. These messages are a rich source of information for companies for sentiment analysis or opinion mining. There is however a great obstacle for analyzing the messages directly: as the company names are often ambiguous (e.g. apple, the fruit vs. Apple Inc.), one needs first to identify which messages are related to the company. We first present simple techniques that make use of company profiles, which we created semi-automatically from external Web sources. Our advanced techniques take ambiguity estimations into account and also automatically extend the basic company profiles through active learning from the Twitter stream itself. We demonstrate the effectiveness of our methods through an extensive set of experiments. We also present TweetSpector as a working prototype for entity-based classification of tweets. (3) Entity Profiling and Applications: Entity profiling is the problem of constructing a compact representation (profile) of an entity, which summarizes the various mentions of an entity. We focus on constructing entity profiles to user and location entities, and show applications that make use of such entity profiles. (a) User-Entity Profiles on Social Networks: Users through their activities on social networks leave traces of their personalities. With the advances of content mining and modeling techniques, it should be possible to profile an user entity from his social network content. In this work we explore various techniques for summarizing a user’s presence on different social networks. We show that one of the advantages of maintaining user profile is to provide the context for understanding the short texts, and help in better understanding of microposts. Additionally, we present TripEneer: Travel plan recommendation application based on user and location entity profiles. (b) Social and Sensor Data Fusion of a Location-entity in the Cloud: As mobile cloud computing facilitates a wide spectrum of smart applications, the need for fusing various types of data available in the cloud grows rapidly. In particular, social and sensor data lies at the core in such applications, but is typically processed separately. This work explores the potential of fusing social and sensor data, related to a location entity, in the cloud. We present a travel recommendation system that is built upon a conceptual framework. This framework allows to blend the heterogeneous social and sensor data for integrated analysis, extracting weather-dependent people’s mood information from Twitter and meteorological sensor data streams. This thesis through these contributions for linking entities on the web makes a promising step towards realizing entity-oriented view of (Semantic) Web.