Probase : a Universal Knowledge Base for Semantic Search

We demonstrate a prototype system that showcases the power of using a knowledge base (Probase) for search. The goal of Probase is to enable common sense computing, and its foundation is a universal, probabilistic ontology that is more comprehensive than any of the existing ontologies. Currently, it contains over 2.7 million concepts harnessed automatically from a corpus of 1.68 billion web pages. Unlike traditional knowledge bases that treat knowledge as black and white, it supports probabilistic interpretations of the information it contains. The probabilistic nature also enables it to incorporate heterogeneous information in a natural way. Besides the system, we also demonstrate two applications, i) semantic web search and ii) understanding and searching web tables, that are built on top of the Probase framework. They indicate that a little common sense goes a long way: machines can be made more intelligent if it has access to the right knowledge. 1. THE KNOWLEDGE BASE We demonstrate Probase, an ongoing project that focuses on knowledge acquisition and knowledge serving. In the age of information explosion, there is a pressing need to enable machines to better understand natural human language. The Probase project shows that such understanding might be made possible if we can harvest “general knowledge” or “common sense” from the enormous amount of data that is available to us, and inject them into computing. The question is then, what is general knowledge or common sense, and can machines grasp common sense? It is certainly difficult for machines to master the entire body of general knowledge. However, it is still possible to give machines certain common sense, and it turns out that a little common sense goes a long way. For example, for human beings, when we see “25 Oct 1881”, we recognize it as a date, but the majority of us do not know what the date is for. However, if we had a little more context, say it is embedded in the text “Pablo Picasso, 25 Oct 1881, Spain”, we would have guessed (correctly) that it is Pablo Picasso’s birthday. Human beings are able to do this because we possess certain common sense, and in this case, “one of the most important dates associated Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, to republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Copyright 200X ACM X-XXXXX-XX-X/XX/XX . $5.00. with a person is his birthday. ” Take natural language processing as another example. Humans do not find sentences such as “animals other than dogs such as cats” ambiguous, but machine parsing can lead to two possible understandings: “cats are animals” or “cats are dogs. ” Common sense tells us that cats cannot be dogs, which renders the second parsing mostly improbable. It turns out that all the common sense needed in the above two cases are about concepts (e.g., persons and animals), instances (e.g., Pablo Picasso, cats, and dogs), attributes (e.g., birthday), and values (e.g., 25 Oct 1881). It is not impossible to build a knowledge base that encode such common sense. Developing countries area = 3,287,263 sq km population = 1.1 billion gdp = $3.57 trillion Markets Emerging Markets European Markets