ISSN 2186-7437

NII Shonan Meeting Report

No. 2014-15

NII Shonan Meeting Report
Ken Satoh
Yuji Matsumoto
Randy Goebel
November 27–30, 2014

National Institute of Informatics
2-1-2 Hitotsubashi, Chiyoda-Ku, Tokyo, Japan

Towards Explanation Production Combining
Natural Language and Logical Reasoning
Organizers:
Ken Satoh (National Institute of Informatics, Japan)
Yuji Matsumoto (Nara Advanced Institute of Science and Technology, Japan)
Randy Goebel (University of Alberta, Canada)
November 27–30, 2014

Meeting motivation and background
The recent application of high performance natural language processing (NLP)
systems has shifted from general question answering to more targeted application domains (e.g., medical diagnosis, epidemiology, legal reasoning, intellectual
property management). Examples include the WATSON project by IBM andthe Todai Robot Project by NII. But these recent systems have focused more on
engineering problem solutions, and less so on general architectures for building
domain models that support more general representation and reasoning within
the specific application domains. For example, the WATSON architect uses anassociative pattern matching process which supports high performance, but isdifficult to follow for humans, and thus prevents easy collaborative problemsolving with humans. Our workshop proposal is focused on the development
and integration of modern natural language processing tools to support the development of systems with methods of hypothesis management and explanation.
In this way, humans can collaborate with such systems to not only understandthe use of representation to create answers, but also support incremental supervised machine learning. The purpose of the proposed meeting is to gather
a group of researchers in related areas, such as natural language processing,information extraction, and logical reasoning, in order to formulate approaches
to combining these areas to achieve the goal of machine-human collaboration in
high-performance natural language-based domain interactive reasoning systems.
A research agenda for such a meeting would include a variety of topics, including language-based information extraction (e.g., open information extraction),
reasoning architectures based on abduction and hypothesis management, natural language entailment, and inductive learning. Specific challenges include thecapture and use of legal documents, in order to answer legal questions. In thiscontext, the creation of legal judgements involves not just answers (e.g., guilty,
not guilty), but articulation of elaborate explanations supported by interpretation of legal statues and regulations. To construct such logical explanations, weneed to identify information relevant to the questions, ensure that informationis transformed to or directly represented in formal representations that support
the contraction of explanations. In this phase, we need an NLP analysis of1

question and information retrieval technique based on the analysis. Subsequent
steps include the development of entailment testing methods, to decide whether
and how (or how not) a question is entailed by the logical representation of
the extracted domain information. The general development of NLP techniques
for this kind of subsumption task involves ontological manipulation, relation
extraction, and logical reasoning. Because of this diversity of requirements,
our workshop will need to include researchers from information retrieval, information extraction, question-answer, general natural language processing, andlogical representation and reasoning. With this context, the goal is to develop
a basis for reliable system which produces an explanation of systems behaviour,and be incrementally constructed and improved to advance their level of performance in a variety of appellation domains. Possible topics of the meeting
include, but are not limited to:• Natural Language Processing for Information Retrieval• Information Retrieval for Complex Logical Formulae• Natural Language Processing for Textual Entailment• Natural Language Processing for Subsumption Testing• Ontology Research for Subsumption Testing• Combining Logical Reasoning and Natural Language Processing• Feedback from Logical Reasoning to Solve Disambiguation

Individual perspective on summary of workshop
themes
The summary of individual perspectives of the workshop impact are included
below, and summarized by a Wordle in Figure 1.
Overall, the workshop provided a good balance of theoretical and practicalwork, and discussions (as indicated below) are largely about continuing to discuss how to keep a balance in research that exploits computational linguistics
and machine learning, from pragmatic system building to achieve a variety of
information question answering and summary tasks.
The individual summaries here are presented in the order in which they
where received.

Randy Goebel, University of Alberta
Our work on the entailment tasks of COLIEE and NTCIR has shown promising
results, produced with a combination of modern NLP semantic parsing techniques, coupled with machine learning methods for knowledge extraction fromlarge text corpora.
Future work will focus on at least two new themes: 1) identification of thelanguage phenomena whose improper treatment explains the failure on some ofthe entailment tasks (e.g., faulty anaphoric reference resolution, no connection
between legal terms buy and sell.), followed by elaboration of the entailment

2Figure 1: Shonan Workshop Wordle Summary
process to handle such cases; and 2) further work on the development of abstract rules for domain specific knowledge (e.g., in the legal and medical domains), which can be used both to improve knowledge extraction in support ofentailment, and guide abductive explanation of alternative entailments.

Chitta Baral, University of Arizona
This meeting has been very useful and gave me many ideas about future research directions in collaboration with other participants. I have already briefly
discussed with some of the other participants but not all of them.
1. Developing a version of COLIEE with shorter text similar to text in Winograd Schema Challenge. The focus would be to test how well systems
capture various logical intricacies in legal reasoning.
2. Randy mentioned some alternatives to the Winograd Schema Challenge.
It would be worth pursuing this.
3. Ido, myself, and Kentaro Inui discussed collaborating and co-ordinating
extraction of knowledge from text.
4. Yusuke Miyao and I discussed on collaborating on developing versions of
NL2KR and kparser for the Japanese language.
5. I like the idea of developing a open-source Watson that was mentioned
by Yuzuru Tanaka. Developing a multi-national effort on this would be a3

good idea.
6. It would be interesting to use the math formula search work of Akiko
Aizawa to solve numerical problems described in text by hunting andextracting the needed mathematical formulas from text (or text books).
For example, it would be interesting to develop a system that can solve
numerical Physics problems. The system will not be hardcoded with the
Physics concepts but rather it (after it gets the problem) goes and finds
the needed Physics formulas and concepts from text and then uses themto solve the problem that is asked.

Erich Schweighofer, University of Vienna
In law, the standard knowledge tool should become more supportive and intelligent. With the semantic legal knowledge system, a concept exists for major
improvements. Research using knowledge representation and knowledge acquisition techniques would improve the tool box of knowledge engineering in thelegal domain and would help to solve the present knowledge crisis in the legal system. Research aims: improve the semantic knowledge model of legal
system, improve and update representation using tools of document categorisation, semi-automatic generation of thesaurus descriptors, automatic generalof hypertext links, automatic generation of temporal relations, ranking, semiautomatic generation of summaries of documents and automatic translation of
documents. International law (European law) is proposed as domain area.

Ido Dagan, Bar-Ilan University
I propose two types of practical next steps that can foster progress in this area,and possibly help bringing together researchers coming from different disciplines.
Inference and explanation benchmarks:• Set up a benchmark dataset (or datasets) for relevant inference tasks overtexts
• Desirably, the task should be taken from a real applied scenario (vs. a
synthetic task like the Winograd schema challenge)• In addition to the required inference decision, specify precisely also a taskof providing explanations for the inference. Here substantial work should
be done, since the notion of explanation is currently quite vague and nonconsensual. This should include clear specifications of the required explanations, as well as a specification of an evaluation measure for explanations
provided by systems
Promote work on knowledge acquisition and sharing of acquired knowledge
resources:• As was evident in the meeting, knowledge acquisition is a major bottleneckfor robust inference• Possibly specify novel benchmarks for knowledge acquisition tasks. Thisis quite challenging, as direct evaluations of knowledge effectiveness are4

quite rare, and often depend on a particular non-standard system and are
therefore hard to replicate.
An example of knowledge evaluation was given in Kentaro Inuis presentation,
where the impact of acquired scripts knowledge was measured via an ablation
test (measuring the marginal impact of the knowledge on the performance of
the inference system).
Similarly, a benchmark could provide one or desirably several standard systems that rely on knowledge, and then asking benchmark participants to provide
knowledge bases and measure their marginal impact on the provided systems.
• Encourage sharing of acquired knowledge resources, via standard repositories and formats• Encourage projects, possibly joint ones, on acquisition of inference knowledge• A far-fetching inspiration could be the Genome project, which has been ahuge joint knowledge acquisition effort.

Günter Neumann, DFKI
Interactive exploratory search:
Our future research in the area of exploratory interactive search will exploit
text inference for cross-sentential and space and time aware open relation extraction in order to identify more hidden relations and event structures. We will
extend our multilingual open information extraction approach to cross-linguality
by exploring Machine Translation and alignment-based strategies. Furthermore,
we will investigate strategies that go beyond isolated topic graphs in order tosupport sharing topic graphs between different users, compare and merge topic
graphs, and to monitor the development of topic graphs over time.

Bob Kowalski, Imperial College
The goal that I propose for future research is the generation of logical representations of natural language texts. The resulting logical forms can be usedboth for deductive and for abductive reasoning , as well as for input to inductive reasoning systems. Entailment and explanation would be two among manydifferent applications. I believe that much of the work that has been done on
NLP can be used to support this task. In particular, it can be help to identify
predicate argument structure, which is the atomic building block of logical representations. I also believe that machine learning can be applied to this task.
Experts can annotate a training set of natural language texts with the information needed to generate the logical representation of the text. The training set
can then be used to learn how to generate the logical representation of new texts.
The bottleneck in this proposal is the failure of logicians to agree upon a logic
that is adequate for this task. However, I believe that there is overwhelming evidence that logic programming, with its focus on the distinction between simple
conclusions and more complex conditions, provides a suitable formalism for thispurpose. In this respect, it resembles the condition-action rules of production
systems, which have been widely used a model of human thinking. Abductive

5

logic programming is a significant extension of ordinary logic programming, and
it is especially well-suited for representing natural language meanings. Abductive logic programs combine ordinary logic programs for representing definitionsof predicates, with (undefined) abducible predicates, which can have associatedprobabilities. Abducible predicates can be constrained by integrity constraints,
which are similar to the obligations and prohibitions that are often be found inlegal texts. There have been many applications of ordinary logic programming
to the representation of legal texts, especially to legislation. I believe theseapplications can provide a sound foundation on which to build systems for understanding more informally written texts. I also believe that there is further
evidence for the use of abductive logic programming, based on my analysis bothof English language texts that are designed to be easy to understand, and on
the advice given by English scholars about how to write English texts that areeasy to understand. One feature of such advice, which can usefully be exploitedwhen attempting to generate logical representation of texts, is that sentences
should start with old, familiar information and end with new information. The
new information at the end of one sentence can serve as the old information at
the beginning of the next sentence. I believe that this feature of natural language texts can help a computer system to disambiguate text, by considering
the logical representation of paragraphs instead of individual sentences.

Bart Verheij, University of Groningen
At this workshop, I hoped to learn about similarities and differences between
two kinds of research:• Research focusing on data-driven techniques, typically involving numericestimates of what is relevant for a problem, e.g., in terms of statistics• Research focusing on knowledge-modeling techniques, typically involving
qualitative representations of what is relevant for a problem, e.g., in termsof logic
This hope was fulfilled, although it has been easier to see differences thansimilarities.
Differences
Data-driven research emphases:
1. Scalability
2. (Big) data/corpora
3. Simple structures and relations (e.g., correlations, triples)
4. Performance evaluation
5. Natural language
6. Approximate answers
7. Statistics

6

Correspondingly, knowledge-modeling research emphases:
1. Representational adequacy
2. Examples/cases
3. Complex structures and relations (e.g, arguments, logical models, scripts/schemes)
4. Normative evaluation
5. Formal language
6. Correct answers
7. Logic
Similarities
Both kinds of research had some version of the following:• Construction of hypothetical answers, interpretations, explanations using
background knowledge• Testing of hypothetical answers, interpretations, explanations using evidence (sometimes supporting, sometimes attacking a hypothesis)• Selection of hypothetical answers, interpretations, explanations using preferences
I like to think of these in terms of the language of argumentation, in which hypotheses are interactively constructed, tested and selected, much like in science(where argumentation is a tool for discovery) and in law (where argumentation
is a tool for problem solving, and also for discovery). Argumentation approaches
combine deductive, inductive and abductive reasoning techniques.
Future research
In my talk, I emphasized that there are questions with simple answers (e.g.,
What is Vincent van Gogh’s country of birth?: The Netherlands) and withquestions with complex answers (e.g., Is the suspect guilty of the crime, andwhy?: Some argument justifying a scenario that explains what has happened).
IBM’s Watson has focused on questions with simple answers. In the present
state of the art of data-driven technology (such as Watson’s) that is to be expected, as such technology has been focusing on simple structures and relations.
In order to extend the state of the art to the answering of questions with complex answers (plans, explanations, arguments, interpretations, configurations),
new technology must be developed that extends on what is now possible. The
time is ripe for the incorporation of knowledge-modeling technology, which havebeen developed for the handling of complex structures and relations.
Hence, in future research, I would like to see the gradual integration ofdata-driven and knowledge-modeling technology. On the data-driven side, thisrequires a gradual extension of the complexity of the structures and relations.
On the knowledge-modeling side, this requires a gradual change from being
example-driven to (also) being data-driven. Intermediate representations (such7as parse trees, argument/rule schemes, scenario schemes) can help connect unstructured data with overstructured logic.
For me personally, the integration of data-driven and knowledge-modelingtechnology would be based on an argumentation perspective that combinesprobabilistic approaches (as relevant for data-driven research) with logical approaches (as relevant for knowledge-modeling approaches). Application domainsof discovery in science and the law would be excellent test beds, and both would
add social value, if successful.
Two kinds of input will lead to synergetic advantages:• Theoretical research: the study of logic, probability theory and other relevant mathematics guarantees clarity, precision and normative correctness.
• Engineering research: the design and actual building of systems is necessary to establish what can and cannot be achieved, and why.
An important goal would be to develop a theoretically well-founded annotated data set (relevant for scientific and legal discovery) that can be used as anormatively correct golden standard for further research.

Yu Asano, Hitachi Central Research Labs
As future work, I would like to consider the following things:
1. How to build a lexicographic vocabulary automatically
2. Introduction of adverbs as a new word class. Probability or fuzzy logic
may be used to defined adjective words, which is ambiguous words (e.g.
gradually, rapidly)
3. Mapping natural language expressions to corresponding intermediate language, which we proposed.

Satoshi Tojo, JAISTThe complication of notation of modal logic is annoying. Especially in the caseof dynamic epistemic logic (DEL), the description of revised model is further
messy. Because DEL lies still in mathematical community, there should be a
gap towards the pragmatic programming. My future tasks are:
1. to show more convincing examples; those which I have shown, e.g., threewise men, sum and product, and so on are solvable in other ways. In
addition, they are purely logical puzzles far from real-world problems. We
need to show the ones that are unique to DEL and more practical.
2. I myself have tried to be a bridge between math community and AI community. One method I recognized this time is to show DEL on computers,i.e., to visualize the accessibility to demonstrate how the belief revision
works. I’m now considering to implement a comprehensible education
system.

8

Yuzuru Tanaka, Hokkaido University
As a focused future research theme, I would like to propose the R&D on open
source Watson-like systems through the intensive and international collaboration of natural language processing and logical reasoning research communities.
The current IBM Watson is still based on the shallow reasoning. Few details
are available about reasoning architecture. It is focusing on business applications
including the support of the negotiation between medical doctors and insurancecompanies, and the provision of alternative advices to medical doctors.
There are strong needs for open source Watson-like systems based on deeper
reasoning. We need to establish an open innovation framework for the R&D of
the required technologies. We should focus more on the support of academic
R&D process and the support for patient empowerment.
The R&D toward this direction may have the following milestone subgoals.
1. Fact retrieval IBM Watson for Jeopardy challenge may corresponds to thisstage.
2. Prediction of the research trend over the next decade This is practicallyuseful for research policy making.
3. Finding a proof of h ⇒ t. The answer may be a proof process: h → t1 →t2 → · · · → t, which explains this reasoning. Each ti → tj is extracted
from some document in the document database.
4. Finding both requisites r and a proof of h, r ⇒ t, the answer may be aset: {r|h, r1 → t1 , t1 , r2 → t2 , t2 , r3 → t3 , t3 , · · · tn−1 , rn → tn }. Abductive
inference will give those requisites.
Stages from (1) to (3) may work for hypothesis checking and its explanation,
whereas the stage (4) may be necessary for the hypothesis making. They may
support the whole research process consisting of the hypothesis making and the
hypothesis checking.

Nguyen Le Minh, JAIST
1. I would like to focus on analyzing a sentence to a logical form representation and its application to textual entailment recognition (RTE) for thelegal text. Recognizing temporal and event in the legal sentences will beconsidered in analyzing a legal sentence to its logical form representation.
2. Sentences in the legal domain are very long and complex, methods for
dealing with those sentences are considered in our future works.
3. Graph knowledge is the one of the possibility for exploiting in my future
work. This will be used for improving semantic parsing accuracy in thelegal domain.

Akiko Aizawa, NII
In our work, we addressed several key issues to support users to retrieve, understand, and utilize mathematical information. Our approach is based on the9identification of links between mathematical expressions and their natural language descriptions in a document. Such associations enable us to semanticallyenrich math formulae and enhance conventional math retrieval and browsing
systems. In my talk I also introduced our three-years struggle to establish an
evaluation platform for math information retrieval in NTCIR-11 Math-2.
Although our current target is primarily focused on mathematical expressions, many of the techniques used in our system came from natural language
processing. In addition to these conventional methods, we also developed ascalable indexing scheme suitable for large-size labeled trees (with variables).
I hope such a math-specific method will be also useful for exploring semantic
structure of text.
So far, I have been mainly working on math retrieval systems. However,through the discussions at this Shonan-meeting, I strongly recognized the necessity of moving forward and start thinking about extracting math-related
knowledge from target documents. Just collecting and categorizing symbolized
concepts in a specific field would reveal some domain knowledge. Another interesting viewpoint is to investigate in detail the difference of tree structures
obtained from mathematical formulae and natural language sentences. Thismight help to learn the process of generalizing text into abstract rules.
Although converting mathematical formulae in a document into executable
codes (for Mathematica or Maple) is a very hard task, it should be consideredsometime in the future. As a starting point, translating natural language text
into arithmetic calculation would be an interesting challenge, particularly when
it is applied to tax law descriptions (suggested by Prof. Ken Sato).

Yuji Matsumoto, NAIST
Through the precise and deep analysis of (complex) sentences, I like to move
forward to analyzing documents of several areas for extracting knowledge andvaluable information. The targets will cover:• Retrieval and summarizing technical documents, such as technical papers, from various perspectives, e.g., background, methodology, new results/findings, etc.
• Finding relationship between different documents/technical papers.
• classifying people’s opinions and their reasons.
• Construction of lexicons that cover multi-word expressions, syntactic patterns, relationship between expression within a languages as well as crosslanguages.
• Development of tools for handling lexicons and obtained knowledge forkeeping and showing them.

Bernardo Magnini, FBKIn the seminar I have learnt interesting research activities carried on by several
research groups, of which I was not fully aware. I have found particularlyinteresting the large-scale acquisition of resources for common-sense reasoning

10

(e.g. scripts) and the use of such resources for abductive reasoning for generating
explanations.
As a general topic for future research I would focus on the relation between
knowledge automatically extracted from text using open domain Information
Extraction, typically represented as textual graphs, and background knowledge,
typically represented with some formal language. While the current state-of-art
in open domain IE does not allow to produce formal representations of enoughquality and coverage, I can see interesting room in the next years for coupling
the two representation levels. I think it would make sense to develop mixed
query/retrieval schemas, where partially structured information (e.g. entailment
graphs) is used together with structured knowledge (e.g. description logic) inorder to provide more informative explanations of reasoning processes.
In this framework, I would be interested to pursue collaborations in thecontext of the CREST and EU research programs.

Kentaro Inui, Tohoku University, JapanI share the view that intelligent computational systems must have the capability of explaining their outputs. Producing explanations is important not only
for sophisticated applications which inherently need to provide the users with
explanations but also for researchers’ better understanding of how their computational models behave and what is required to make them better. This is also
one of the motivations behind my group’s exploration for the abduction-based
approach.
One issue I think is important in this movement toward explanation production is how to evaluate the ”goodness” of an explanation and how to compare
different methods. Good explanations to a given situation may not be uniqueand different disciplines (or even different individual researchers) may have different models of explanations. This provisional problem reminds me of the issueof evaluating studies in language generation. In language generation, it is not as
straightforward to evaluate outputs as it is in language analysis partly because
language generation tasks sometimes have open-ended ”appropriate” potential
outputs for a given input and thus it is not easy to build ”gold standard”
datasets. Language analysis tasks, on the other hand, tend to be decomposable
into tasks of label or structure prediction, where the correct answers are usuallyunique and not as arguable as in language generation. Research for explanation
production involves the same issue in nature and may raise even more complex
issues because it is more content-oriented and knowledge-dependent.
I think this is an intriguing issues for which researchers with different backgrounds can cooperate together. NLP, logic, knowledge modeling, argumentation theory and logic of thought are all indispensable pieces for addressing thatissue. How describe and annotate explanations humans produce. How to collect humans’ explanations. How to compare our systems’ outputs with humans’
explanations. I think these are important questions which drive our researchforward.

11

Overview of Talks
Computational Logic, the Language of Thought and Natural Language
Robert Kowalski, Imperial College, United Kingdom
Formal Logic is a natural candidate for representing computer-intelligible
knowledge extracted from natural language texts on the WWW. I will argue
that the logic of natural language texts is normally not visible, but is hidden
beneath the surface, and that it can be uncovered more easily by studying
texts that are designed to be as clear and easy to understand as possible. I will
support my argument in two ways: by giving examples of English language texts
and their hidden logic, and by interpreting guidelines for English writing stylein computational logic terms. I will also argue that kind of logic that is mostuseful for representing natural language is both simpler and richer than classicallogic. It is simpler because it has a simpler syntax in the form of conditionals,
and it is more powerful because it distinguishes between the logic of beliefs andthe logic of goals.
Keywords: Computational Logic, English writing style, Knowledge representation.

Arguments for Structured Hypotheses: A Logico-Probabilistic
Perspective
Bart Verheij, University of Groningen, The Netherlands
Some questions have answers with a simple structure. For instance, thequestion ”What is Vincent van Gogh’s country of birth?” has the answer ”TheNetherlands”. Other questions require answers with a more elaborate structure.
For instance, although the question ”Is the suspect guilty of a crime?” canbe answered with a simple ”yes” or ”no”, additional structure in the form ofarguments for and against the legally relevant scenarios is needed. Each scenario
provides a hypothetical answer to the guilt question. Some scenarios are bettersupported by the evidence than others. In the talk, a theory of arguments
for structured hypotheses is discussed that uses classical logic and probability
theory as normative framework. Possible answers to questions take the form ofstructured hypotheses. The logico-probabilistic argumentation framework sheds
new light on the formal semantics of argumentation, in a way that combines
logic-based knowledge technology with probability-based data analysis.
Keywords: Argumentation, Inference to the Best Explanation, Artificial Intelligence and Law, Combining Logic and Probability Theory.

EXCITEMENT: EXploring Customer Interactions throughTextual EntailMENT
Bernardo Magnini, Ido Dagan, Guenter Neumann, Sebastian Pado, Fondazione
Bruno Kessler, Italy
EXCITEMENT (http://www.excitement-project.eu) is a 3-year research project
funded by the European Commission. The main topic of the project is identi12

fying semantic inferences between text units, a major language processing task,needed in practically all text understanding applications. On the industrial side,
EXCITEMENT is focused on the text analytics market and follows the increasing demand for automatically analyzing customer interactions. A major resultof the project is the release of the EXCITEMENT Open Platform (EOP). The
platform aims to automatically check for the pres-ence of entailment relations
among texts. It is based on a modular architecture and provides support for thedevelopment of algorithms that are language independent to a high degree. The
result is an ideal software environment for experimenting and testing innovative approaches for textual infer-ences. The EOP is distributed as open source
software (http://hltfbk.github.io/Excitement-Open-Platform/).
Keywords: semantic inferences, textual entailment, inference platform.

Decomposing Semantic Inferences
Bernardo Magnini, Fondazione Bruno Kessler, Italy
Textual Entailment (TE) has been proposed as an applied framework to
capture major semantic inference needs across applications in Computational
Linguistics. We think that crucial progress may derive from a focus on decomposing the complexity of the TE task into basic phenomena and on their
combination. In this talk, we carry out a deep analysis on TE data sets, investigating the relations among two relevant aspects of semantic inferences: thelogical dimension, i.e. the capacity of the inference to prove the conclusion fromits premises, and the linguistic dimension, i.e. the linguistic devices used to accomplish the goal of the inference. We propose a decomposition approach overTE pairs, where single linguistic phenomena are isolated in atomic inference
pairs, and we show that at this granularity level the actual correlation between
the linguistic and the logical dimensions of semantic inferences emerges and canbe empirically observed.
Keywords: semantic inferences, textual entailment, linguistic phenomena

Natural Language Knowledge Graphs: Open-IE meets Knowledge Representation
Ido Dagan, Bar-Ilan University, Israel
Formal knowledge representation schemes are typically limited to pre-definedstructures and predicates. Conversely, Open Information Extraction (OpenIE) represents arbitrary propositions occurring in text, but lacks a consolidating canonical structure and is limited to simple predicate-argument tuples. I
will outline a proposal towards a more powerful knowledge open representation
scheme, which could cover knowledge beyond the typical scope of pre-specified
knowledge representation. First, we propose extracting complex and implied
propositions and abstracting semantically-relevant information. Second, we propose adding a structure over the set of extracted propositions via relevant semantic relationships. We first focus on the textual entailment relation, which
consolidates semantically equivalent propositions and induces a useful specificto-general hierarchical structure. I will review initial research activities along
the above mentioned goals.
13Keywords: natural language processing, knowledge representation, textual
inference, textual entailment, open information extraction

Identifying the Tradeoffs in Textual Entailment: Deep Representation versus Shallow Entailment
Randy Goebel, University of Alberta, Canada
Much research on natural language understanding and processing (”NLP”)
has focused on how to transform natural language to formal logics, in which casethe problem of text entailment becomes that of logical entailment. Despite avariety of approaches to the transformation of language to logic, even the mostsophisticated (e.g., Montague’s higher order intensional logics or Steedman’s
combinatory categorial grammar) leave unresolved foundational challenges like
context and dialogue. And though these transformations are tightly coupledwith formal mechanisms for inference, those methods themselves are often difficult to implement. Current text entailment focuses on building or learningmodels of verb cases, concept identification, summarization, and information
extraction. We consider consider some measure alternatives for the tradeoffs,
and whether they are necessarily empirical, or can exploit some foundational
principles of NLP representation theory.
Keywords: entailment,formal,representation,inference Slides Summary

Learning to Parse Legal Sentences to Logical Form Representations
Nguyen Le Minh and Akira Shimazu, Japan Advanced Institute of Science andTechnology, Japan
In this talk, we would like to present our framework for dealing with the
problems of understanding legal sentences. Our framework is divided into threemajor steps: logical parts recognition, logical part grouping, and semantic parsing. In the first phase, we model the problem of recognizing logical parts in law
sentences as a multi-layer sequence-learning problem, and present a CRF-basedmodel to recognize them. In the second phase, we propose a graph-based method
to group logical parts into logical structures. We consider the problem of finding
a subset of complete subgraphs in a weighted-edge complete graph, where eachnode corresponds to a logical part, and a complete subgraph corresponds to alogical structure. For the final step, we would like to report our recent works
and the state of the art semantic parsing models for general domains. We also
discuss the potential of exploiting current semantic parsing models for simple
law sentences.
Keywords: Legal text processing, semantic parsing, logical parts recognition

Parsing Long and Complex Natural Language Sentences
Yuji Matsumoto, Nara Institute of Science and Technology, Japan
While syntactic analysis of natural language sentences has shown a remarkable progress in past decades, there are still some hindrances to further improvement. Sentences in scientific or legal areas often have very complex structures,14

mainly due to long coordinate structures and/or complex sentence patterns.
Although most of the current machine learning-based syntactic parsers use local features to decide phrase structures or word-to-word dependencies, globalfeatures or resources that make use of long distance dependencies are necessary to handle complicated linguistic phenomena. In this talk, I will introduce
our on-going project to develop methods and linguistic resources for complex
structures, such as coordination structures and complex syntactic patterns.
Keywords: natural language parsing, coordination structures, complex sentence patterns

Explanation Producing Combination of NLP and Logical
Reasoning through Translation of Text to KR Formalisms
Chita Baral, University of Arizona, USA
Our approach to combine NLP and logical reasoning so that it can produce explanations is based on translating natural language text to appropriate knowledge representation formalisms. In this talk we will discuss two approaches. In the first approach, we will present our semantic parser, available at http://kparser.org , that translates English text to a knowledge graphthat includes ontological and domain knowledge from various sources. The second approach addresses the concern that depending on applications one may
want or need translations of natural language text to different knowledge representation formalisms. We will present our NL2KR platform (available at
http://nl2kr.engineering.asu.edu ) that allows the development of translation
systems by giving examples of translations (i.e., a training set) and an initialdictionary of words and their meaning given as lambda calculus expressions.
The work that will be presented is done by Chitta Baral and his students.
Keywords: Natural Language Processing, Semantic Parser, Lambda Calculus

Reading Between the Lines
Kentaro Inui, Tohoku University, Japan
Based on Scalable and Trainable Abduction and Large-scale Knowledge Acquisition The idea of modeling semantic and discourse analysis based on logicalabduction goes back to Hobbs et al.(1993)’s influencing work: Interpretation as
Abduction. While the approach has many potential advantages, no prior work
has successfully built abductive models applicable to real-life problems chiefly
due to the lack of knowledge and computational costs. However, the game isdrastically changing. Recent advances in large-scale knowledge acquisition fromthe Web may resolve the knowledge bottleneck. We show that the computational cost of first-order abduction can be considerably reduced with the liner
programming techniques, which then enables the supervised training of abduction. Given these movements, we believe that a number of intriguing issues will
emerge from the resumption of the study for abduction-based modeling of NLP
tasks with a fast reasoner and large-scale knowledge resources. We will present
recent insights gained from experiments on the Winograd Schema Challenge.

15

Keywords: discourse analysis, abduction, knowledge acquisition, coreference
resolution, Winograd Schema Challenge

Towards Explanation Production of Yes/No Questions in
Multiple Choice Bar Exam
Ken Satoh, National Institute of Informatics, Japan
We will present an approach toward explanation production for multiplechoice bar exam where we give an explanation why a branch in multiple choices isfollowed by the articles or precedents in civil law. We use PROLEG (PROLOGbased LEGal reasoning system) for a reasoning part which we developed forreasoning about ultimate fact theory and try to make a connection between
PROLEG predicate and an element in a parsed tree of bar exam sentences by
NLP.
Keywords: Legal Reasoning, PROLEG, logic programming, parsed tree

How to Promote the R&D on Open Source Watson-like
Systems based on the Combination of Natural Language
Processing and Logical Reasoning?
Yuzuru Tanaka, Hokkaido University, Japan
In 2013, JST launched the CREST program on big data applications. It
has picked up six projects during the first two years, and will pick up anothercouple of projects next year. As one of the focused research areas for the callfor proposals next year, the program is now planning to pick up the NLP basedknowledge acquisition from published research papers in logical representation,
the construction of a large scale knowledge base, and the automatic reasoning
with this knowledge base for knowledge discovery and question answering. This
research area became stimulated by the success of IBM Watson in ”Jeopardy!”
This research direction may be considered a a scalable extension of the dreamsof Japanese fifth generation computer project in 90s with NLP interface. As
the program officer of the JST CREST program, I would like to ask a question
on how to promote the R&D on open source Watson-like systems based on the
combination of natural language processing and logical reasoning.
Keywords: big data applications, open-source Watson like systems, researchpromotion, knowledge discovery from published papers

Math Formula Search
Akiko Aizawa, National Institute of Informatics, Japan
Mathematical formulae in natural language text sometimes represent formalized concepts and relations that can be processed by computers. However,in actual documents, most formulae are expressed as noisy, ambiguous, andinsufficient representations. In the past, we explored how to deal with such’informality’ of formalized and abstracted relations for efficient semantic search.
This reveals to require elaborative analysis of surrounding natural language text
as well as efficient approximate tree search enhanced with variable unification.
16

In this presentation, we briefly introduce up-to-date techniques for math formula search and also explore further research directions to connect such efforts
to the manipulation of semantic structure embedded in natural language text.
Keywords:natural language processing, math formula search, description extraction, construction of dependency graph

Fact Validation by Recognizing Textual Entailment
Yusuke Miyao, National Institute of Informatics, Japan
We will introduce recent research activities on fact validation, which isalso known as true-or-false question answering. Fact validation aims to prove
whether or not a given statement is true, according to a prespecified set of texts,
such as textbooks and Wikipedia, that are supposed to describe true facts. A
shared task on fact validation has been organized in NTCIR, and its organization scheme and the results of participating systems are introduced. We will
also describe experiments on applying a logic-based textual entailment system
to fact validation, and discuss its advantages and difficulties.
Keywords: fact validation, true-or-false question answering, textual entailment recognition

Identifying Appropriate Concepts for Unknown Words withFormal Concept Analysis
Akihiro Yamamoto and Madori Ikeda, Kyoto University, Japan
In natural language processing, extending many thesauri is time-consuming.
In order to overcome this problem, we propose a method with a corpus. We
assume that extending thesauri should be inserting unknown words into themby finding accurate concepts. We regard the task as classification and use aconcept lattice for it. The method enables us to decrease the time-cost by
avoiding feature selection for each pair of a set of unknown words and a setof unknown words. More precisely, a concept lattice is generated from only a
set of known words, and each formal concept is given a score with the set. By
experiments using practical thesauri and corpora, we show that our methods
can give more accurate concepts to unknown words than other the k-nearest
algorithm.
Keywords: extending thesauri, classification, formal concept analysis

On the Way to Semantic Legal Knowledge Systems
Erich Schweighofer, University of Vienna, Austria
On the Way to Semantic Legal Knowledge Systems
Keywords: Semantics, Legal Ontologies, Logic

17

Knowledge-Intensive Structural NLP in the Era of Big Data
Sadao Kurohashi, Kyoto University, Japan
Texts are the basis of human knowledge representation, including data analysis results and interpretation by experts, criticism and opinions, procedures andinstructions. We have been working on the realization of knowledge-intensive
structural NLP which can extract truly valuable knowledge for human beings
from an ever growing volume of texts, known recently as Big Data. This talk introduces several of our on-going projects concerning knowledge-intensive structural NLP: synonymous expression, case frame and event relation acquisition
from 15G parsed sentences, ellipsis resolution considering exophora and author/reader information, an open search engine infrastructure TSUBAKI, andan information analysis system WISDOM.
Keywords: Knowledge-Intensive Structural NLP, Case Frame Acquisition,
Event Relation Acquisition Slides

Interactive Text ExplorationGünter Neumann, German Research Centre for AI, Germany
Today’s Web search is still dominated by a document perspective: a user
enters keywords that represent the information of interest and receives a ranked
list of documents. This technology has been shown to be very successful, because
it very often delivers concrete web pages that contain the information the user isinterested in. If the user only has a vague idea of the information in question orjust wants to explore the information space, the current search engine paradigm
does not provide enough assistance. The user has to read through the documents
and eventually reformulate the query for finding new information. So seen,
current search engines seem to be best suited for “one-shot search” and do notsupport content-oriented interaction. In my talk, I will present and discuss ourefforts in building highly dynamic, scalable interactive intelligent text content
exploration strategies which supports both, the computer and the human, tointeractively “talking about something.”
Keywords: information search, open information extraction, interactive text
exploration

Agent Communication and Belief Change
Satoshi Tojo, Japan Advanced Institute of Science and Technology, Japan
Communication in agents is not a simple message passing. A rational agent
should send logically consistent contents in the situation. Then, there must be
a communication channel between agents, e.g., an address of the message recipient. Furthermore, the message can be publicly announced, i.e., there can be
simultaneous multiple recipients; otherwise the message passing becomes a personal communication. Finally, the message recipient must adequately maintainthe consistency of their belief, that is, as a result of message passing, the recipient must revise his/her belief to be logically consistent. In this talk, I overview
the various researches concerning logical representation of communication andbelief change, especially in terms of modal logic, where belief change is realized18

by the restriction of accessibility to some possible worlds. Thereafter, I show
some applications of the formalization, such as logical puzzles.
Keywords: agent, communication, belief, dynamic epistemic logic

Explanation Production with Open Data: Approach forQuerying RDF Data by Using Natural Language
Yu Asanao, Hitachi Central Research Laboratory, JapanThe governments of many nations publish vast amount of open data, suchas statistics and white paper. We introduce these activities of open data anddiscuss possibilities to produce explanations by using open data. For example,
an answer on question and answering system ”The population is growing” canbe supported by showing observed statistical data. From this point of view,connecting a natural language expression to its evidence data is a key methodof explanation production. As the first goal for the connection, we propose amethod to inquiry structured data by using natural language like query expression.
Keywords: Open Data,Explanation Extraction, Resource Description Framework, Query Language, Natural Language

Workshop participants• Akido Aizawa, NII, Japan• Yu Asano, Hitachi Central Research Labs, Japan• Chitta Baral, Unviersity of Arizona, USA• Ido Dagan, Bar Ilan University, Israel• Randy Goebel, University of Alberta, Canada• Kentaro Inui, Tohoku University, Japan• Robert Kowalski, Imperial College, United Kingdom• Sadao Kurohashi, Kyoto University, Japan• Bernardo Magnini, FBK, Italy• Yuji Matsumoto, NAIST, Japan• Günter Neumann, DFKI, Germany• Minh Le Nguyen, JAIST, Japan• Ken Satoh, NII, Japan• Erich Schweighofer, University of Vienna, Austria• Satoshi Tojo, JAIST, Japan• Bart Verheij, University of Groningen, The Netherlands• Akihiro Yamamoto, Kyoto University, Japan• Miyao Yusuke, NII, Japan
19

Workshop schedule====================================
November 26 (Wed)15:00 check-in
19:00-21:00 Welcome Reception
21:00-23:00 Night Session
November 27 (Thu)
07:30-09:00 Breakfast
09:00-09:10 Introduction movie of NII Shonan Meeting
09:10-10:30 Brief Introduction of Your Presentation (in 3 minutes)
10:30-11:00 Break
11:00-12:00
Robert Kowalski
Computational Logic and its Relationship with Guidelines for English Writing
Style
Bart Verheij
Arguments for Structured Hypotheses: A Logico-Probabilistic Perspective
12:00-13:30 Lunch
13:30-15:30
Bernardo Magnini, Ido Dagan, Guenter Neumann, Sebastian Pado
EXCITEMENT: EXploring Customer Interactions through Textual EntailMENT
Bernardo Magnini
ecomposing Semantic Inferences
Ido Dagan
Natural Language Knowledge Graphs: Open-IE meets Knowledge Representation
Randy Goebel
Identifying the Tradeoffs in Textual Entailment: Deep Representation versus
Shallow Entailment
15:30-16:00 Break
16:00-17:00
Nguyen Le Minh (joint work with Akira Shimazu)
Learning to Parse Legal Sentences to Logical Form Representations
Yuji Matsumoto
Parsing Long and Complex Natural Language Sentences
17:00-18:00 Discussion
Discussion Question: Can you please note your immediate observations about
20

your original expectations for the workshop, and how they have so far beenchanged or adjusted?
18:00-19:30 Dinner
21:00-23:00 Night Session
November 28 (Fri)
07:30-09:00 Breakfast
09:00-10:30
Chitta Baral
Explanation Producing Combination of NLP and Logical Reasoning throughTranslation of Text to KR Formalisms
Kentaro Inui
Modeling Reading between the Lines Based on Scalable and Trainable Abduction and Large-scale Knowledge Acquisition
Ken SatohTowards Explanation Production of Yes/No Questions in Multiple Choice Bar
Exam
10:30-11:00 Break
11:00-12:00 Discussion
Discussion Questions: From discussions and presentations so far, what are themeasures that could be used to guide the pursuit of the engineering of systems
that provide natural language explanations? Are there underlying scientificchallenges which delimit possible scientific measures?
12:00-13:30 Lunch
13:30-15:30
Yuzuru Tanaka
How to Promote the R&D on Open Source Watson-like Systems based on the
Combination of Natural Language Processing and Logical Reasoning?
Akiko Aizawa
A Linguistic Approach to Math Formula Search
Yusuke Miyao
Fact Validation by Textual Entailment Recognition
Akihiro Yamamoto (Joint work with Madori Ikeda)
Identifying Appropriate Concepts for Unknown Words with Formal Concept
Analysis
15:30-16:00 Break
16:00-18:00 Discussion
Discussion Question: Now at the end of the second day, what concepts haveemerged to be most important so far?
18:00-19:30 Dinner21

21:00-23:00 Night Session
November 29 (Sat)
07:30-09:00 Breakfast
09:30-10:30
Erich Schweighofer
Logic and Semantics in Legal Text Corpora: The Dynamic Legal Electronic
Commentary
Sadao Kurohashi
Knowledge-Intensive Structural NLP in the Era of Big Data
10:30-11:00 Break
11:00-12:00 Discussion
Discussion Question: What major themes have emerged as a result of the meeting?
12:00-13:30 Lunch
13:30-19:00 Excursion
19:00-21:30 Banquet
November 30 (Sun)07:30-09:00 Breakfast
09:00-10:30
Günter NeumannInteractive Text Exploration
Satoshi Tojo
Agent Communication and Belief Change
Yu Asano
Explanation Production with Open Data: Approach for Querying RDF Data
by Using Natural Language
10:30-11:00 Break
11:10-12:00 Wrapping-up
Discussion Question: What are the major themes developed during the meetingthat will impact your future research trajectories (if any)?
12:00-13:30 Lunch
1330-1400 Good-bye(Farewell)====================================

22