Neural-symbolic integration and the Semantic Web

Symbolic Systems in Artificial Intelligence which are based on formal logic and deductive reasoning are fundamentally different from Artificial Intelligence systems based on artificial neural networks, such as deep learning approaches. The difference is not only in their inner workings and general approach, but also with respect to capabilities. Neural-symbolic Integration, as a field of study, aims to bridge between the two paradigms. In this paper, we will discuss neural-symbolic integration in its relation to the Semantic Web field, with a focus on promises and possible benefits for both, and report on some current research on the topic. Approaches in Artificial Intelligence (AI) based on machine learning, and in particular those employing artificial neural networks, differ fundamentally from approaches that leverage knowledge bases to perform logical deduction and reasoning. The former are connectionist or subsymbolic AI systems able to solve complex tasks over unstructured data using supervised or unsupervised learning, including problems which cannot reasonably be hand-coded by humans. Subsymbolic methods are generally robust against noise in training or input data and have recently, in the wake of deep learning, been shown to exceed human performance in tasks involving video, audio, and 1We focus herein on deductive reasoning. Logical inductive and abductive reasoning have also been looked at in the Semantic Web context, e.g. [22,14], but to keep the discussion concise, we have not included them in this treatise. text processing. The latter are symbolic systems that thrive under the presence of large amounts of structured data, including for agent planning, constraint solving, data management, integration and querying, and other traditional application areas of knowledge-based systems and formal semantics. Classical rule-based systems, ontologies, and knowledge graphs that power search and information retrieval across the Web are also types of symbolic AI systems. Symbolic and subsymbolic systems are rather complementary to each other. For example, the key strengths of subsymbolic systems are weaknesses of symbolic ones, and vice versa. Symbolic systems are brittle, i.e., susceptible to data noise or minor flaws in the logial encoding of a problem, which stands in contrast to the robustness of connectionist approaches. But subsymbolic systems are generally black boxes in the sense that 1570-0844/19/$27.50 c © 2019 – IOS Press and the authors. All rights reserved 2 Hitzler, Bianchi, Ebrahimi, Sarker / Neural-Symbolic Integration and the Semantic Web the systems cannot be inspected in ways that provide insight into their decisions (despite some recent progress on this in the wake of the explainable AI effort) while symbolic knowledge bases can in principle be inspected to interpret how a decision follows from input. Most importantly, symbolic and subsymbolic systems contrast in the types of problems and data they excel at. Scene recognition from images appears to be a problem which in general lies outside the capabilities of symbolic systems, for example, while complex planning scenarios appear to be outside the scope of current deep learning approaches. On a more technical level, symbolic and subsymbolic systems differ fundamentally in how they represent data, information, or knowledge. Symbolic systems typically utilize structured representation languages, e.g. stemming from formal logic and the subfield of AI known as knowledge representation and reasoning. Trainable artificial neural networks, on the other hand, typically use representations based on high-dimensional Euclidean space, i.e. real-valued vectors, matrices, etc., and it is by no means obvious how reconciliations between these representation forms can be designed. The complementary nature of these methods has drawn a divide in the rich field of AI. The divide is technical in nature, as symbol manipulation as captured by logical, deductive reasoning, which lies at the core of symbolic approaches, cannot be sufficiently performed using current subsymbolic systems. Moreover, the training to study subsymbolic systems (involving probability theory, statistics, linear algebra, and optimization) differs from symbolic systems (involving logic and propositional calculus, set and recursion theory, and advanced computability reasoning) so strongly that AI researchers tend to find a side of the divide based on their intellectual interests and background. The divide is also cultural in nature, one of mindsets and prior believes, that in the past could sometimes split the academic AI research community by provoking (heated) fundamental discus2The topic is being investigated, of course, and some recent progress is made. E.g., [1] reports on an application of deep learning to planning, and explicitly frames it as work towards bridging the “subsymbolic-symbolic boundary. ” 3It is possible to establish a formal, mathematical bridge in some cases, as e.g. laid out in [31], but so far with limited applicability [3]. sions. The divide is even geographical, where the European Union holds a much higher prevalence of researchers working on symbolic approaches than in the United States. Neural-Symbolic Integration [2,4,28,16], as a field of research, addresses fundamental problems related to building a technical bridge between the symbolic and subsymbolic sides of the divide. The promises from a successful bridging of the divide are plenty. In the abstract, one could hope for bestof-both-worlds systems, which combine the transparency and reasoning-ability of symbolic systems with the robustness and learning-capabilities of subsymbolic ones. As such, integrated symbolicsubsymbolic systems may be able to address the knowledge acquisition bottleneck faced by symbolic systems, learn to perform advanced logical or symbolic reasoning tasks even in the presence of noisy or uncertain facts, and even yield self-explanatory subsymbolic models. This is the promise of added value of neural-symbolic integration research for Computer Science. And also more fundamentally, a bridging may shed insights into how natural (human) neural networks can perform tasks as witnessed by homo sapiens pursuing mathematics, formal logic, and other pursuits that we, introspectively, see as symbolic in nature; this is a basic research problem for Cognitive Science as a discipline. In the following, we will first lay out, in more detail, promises and possible benefits of neuralsymbolic integration research for the Semantic Web. Then we will look at potential benefits of Semantic Web and neural-symbolic integration research for deep learning. Finally, we will also provide brief pointers to some current research going on in relation to this theme. 1. Benefits of Neural-Symbolic Integration for the Semantic Web One of the issues that plagues the Semantic Web (as well as many other fields in Computer Science and its applications) is the knowledge acquisition bottleneck. It refers to the difficult issue of encoding or otherwise storing knowledge, as structured information, for use in Computer Science 4See also http://www.neural-symbolic.org/ Hitzler, Bianchi, Ebrahimi, Sarker / Neural-Symbolic Integration and the Semantic Web 3 applications. The manual encoding of such information, e.g. from human experts’ knowledge, is a very slow and time-consuming, thus costly, process involving both topic experts and knowledge engineers. At the same time, automated methods are a far cry from producing artifacts (e.g., from textbooks, technical documentations, and other written sources) which would be of sufficient quality for use in intelligent systems applications based on logical inference, such as expert systems, or for data curation and integration. The underpinnings of key Semantic Web standards, such as RDF [9] and OWL [29], are explicitly logical, which reflects that Semantic Web applications often rely on high data (and schema/ontology) quality, similar to knowledge bases used primarily for deductive reasoning. The knowledge acquisition bottleneck in the Semantic Web field is very noticeable, e.g., given that the creation of ontologies as well as the creation of high-quality knowledge graphs involves high amounts of human export labour and is correspondingly expensive. The promise of integrated neural-symbolic systems is that they would be capable of both learning and (deductive) reasoning, and thus that they would be able to acquire, through machine learning, knowledge which is of sufficiently high quality to perform deductive reasoning. This anticipated capability directly addresses the knowledge acquisition bottleneck. There is, thus, a promise in this line of work that integrated neural-symbolic systems will lead to – better methods for automated ontology construction, – better methods for ontology population (and, thus, knowledge graph construction), – better methods for ontology alignment, – better methods for assessing the quality of knowledge graph content, and similar major lines of research central to the Semantic Web field. At the same time, integrated neural-symbolic systems carry the promise of being able to perform deductive reasoning – after training – using a (highly parallel) artificial neural network architecture. Consequently, reasoning using such systems can be expected to be extremely fast. This contrasts with traditional deductive reasoning methods, which are usually designed to be provably sound and complete but suffer from long algorithm runtimes. While there has been significant progress on developing highly efficient deductive reasoning engines for Semantic Web content, this remains an issue given ever-increasing availability of data. In fact, the underlying problem is fundamental, as sound and complete reasoning over Semantic Web data necessarily suffers from high computational complexity [30]. Integrated neural-symbolic systems would perform reasoning after training, and presumabl