text: {
    "name" : "1409.2073.pdf",
    "metadata" : {
      "source" : "META",
      "title" : "An NLP Assistant for Clide",
      "authors" : [ "Tobias Kortkamp" ],
      "emails" : [ ],
      "sections" : [ {
        "heading" : null,
        "text" : "Fachbereich 3: Mathematik und Informatik\nBachelor Report\nAn NLP Assistant for Clide\nTobias Kortkamp\nMatriculation No. 2491982\nMonday 26th May, 2014\nFirst reviewer: Prof. Dr. Rolf Drechsler\nSecond reviewer: Dr. Berthold Hoffmann\nAdditional advisors: Dr. Mathias Soeken and Dipl.-Inf. Martin Ring\nar X\niv :1\n40 9.\n20 73\nv1 [\ncs .C\nL ]\n7 S\nep 2\n01 4\nSelbstständigkeitserklärung\nHiermit erkläre ich, dass ich die vorliegende Arbeit selbstständig angefertigt, nicht anderweitig zu Prüfungszwecken vorgelegt und keine anderen als die angegebenen Hilfsmittel verwendet habe. Sämtliche wissentlich verwendete Textausschnitte, Zitate oder Inhalte anderer Verfasser wurden ausdrücklich als solche gekennzeichnet.\nBremen, den 26.05.2014\nTobias Kortkamp\n5 Contents"
      }, {
        "heading" : "1 Introduction 7",
        "text" : ""
      }, {
        "heading" : "2 Basics 13",
        "text" : "2.1 Clojure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 2.1.1 Logic programming with core.logic . . . . . . . . . . . . . . . . . . . . . . . . 14 2.1.2 Tawny-OWL . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 2.2 CoreNLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16"
      }, {
        "heading" : "3 Approach 17",
        "text" : "3.1 Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 3.2 Reconciler . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2.1 Chunking . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 3.2.2 Incorporating text changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 3.2.3 Chunk annotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 3.3 Integration of CoreNLP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 3.3.1 Accessing sentences of a text . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.2 Word maps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 3.3.3 Coreferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 3.4 Building an NLP knowledge base . . . . . . . . . . . . . . . . . . . . . . . . . . 26 3.5 Clide annotations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.5.1 Annotation streams . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 3.5.2 Annotation levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 3.5.3 Annotations provided by clide-nlp . . . . . . . . . . . . . . . . . . . . . . . . 31"
      }, {
        "heading" : "4 Triples 35",
        "text" : "4.1 Triple builders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 4.2 Reifying triples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 4.3 Exporting an OWL ontology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49"
      }, {
        "heading" : "5 Use case: Graph creation from a natural language specification 51",
        "text" : "5.1 Triple walks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55\n6 Conclusion 57"
      }, {
        "heading" : "6 Contents",
        "text" : ""
      }, {
        "heading" : "A Part of Speech Tags 59",
        "text" : "B Installation notes 61\nList of Figures 63\nBibliography 65\n7 1. Introduction\nWhile developing software or hardware natural language texts are part of the process and used to record or specify the system requirements. The problem is to automatically extract the information contained in these texts and make them available for processing. There are some tools that support developers who want to work with and extract information from these specifications. We are missing an approachable way to define simple rules to automatically extract and use information from a text, that is not written in a severely restricted subset of English.\nSome tools like e.g. Cucumber1, a behavior-driven development framework, solve this by only supporting a DSL.2 It provides a DSL called Gherkin, which allows users to write test scenarios that both computers and humans can understand. Scenarios consist of steps and each step is parsed using a user provided regular expression [22]. As a consequence a step’s regular expression is coupled with the specific phrasing that is used in the step definition. A slight variation in its phrasing without updating the corresponding regular expression or adding a new regular expression, might break the scenario because the provided regular expression does not match the step anymore. Ideally, we would like for two steps, with slightly different phrasing but the same information content, to yield the same output and not break the scenario.\nDefining and refining these extraction rules is not a solitary activity, but a collaborative one. By integrating such a system with Clide,3 a development environment with collaboration baked in, we support this aspect from the start.\nClide is a project that was developed as part of Martin Ring’s diploma thesis in 2013 at the University of Bremen. It was originally intended to be a web-based development environment for Isabelle4 only [14, 19]. In contrast with previous Isabelle interfaces, it provides better visualization of the prover’s results than traditional and sequential REPL5-based interfaces through leveraging Web technologies like HTML5 and JavaScript [19]. It has since undergone further development and has evolved to facilitate collaborative editing of documents with support for other languages besides Isabelle [20].\nClide documents are annotated by assistants. It uses an approach called Universal Collaboration where an assistant is seen as just an additional collaborator by the system [20]. While Clide is distributive and asynchronous in nature, it provides an interface that can be used to implement assistants “in a simple, synchronous manner” [20].\n1Available at http://cukes.info/ 2Domain Specific Language 3Available at https://github.com/martinring/clide2 4An interactive theorem prover, available at http://isabelle.in.tum.de/ 5Read Eval Print Loop\n8 A Clide assistant is informed about state changes of Clide’s environment, be it newly opened documents or edits of a document. It can provide domain specific annotations for any parts of a document’s text.\nThis report describes clide-nlp, an NLP6 assistant for Clide. The assistant has the following goals:\n• Create a framework for extracting ontologies from a text, by\n– creating an NLP knowledge base (see Chapter 3), and\n– using simple queries on that knowledge base to extract useful information from the text (see Chapter 4).\n• Provide annotations for interacting with a text to assist in developing of the queries, including showing\n– the semantic graph of a sentence (see Figure 1.1),\n– the coreferences7 of the text, and\n– the ontology extracted from the text (see Figure 1.2).\n• Work in the collaborative environment that Clide provides and keep the ontology and annotations up to date and in sync with text changes.\n• Being one of the first Clide assistants not developed by Clide’s author, clide-nlp should also be seen as a test of the limits of Clide’s current API for developing assistants. Clide’s development continues in parallel with the work on this report.\nChapter 3 introduces the components that constitute clide-nlp and how they interact with each other. Section 3.1 and Section 3.5 discuss how clide-nlp is integrated into Clide.\nBecause of Clide’s interactive and collaborative nature, clide-nlp has to contend with continuously changing text. Section 3.2 discusses a model for providing support for incorporating text changes into clide-nlp’s internal model. The provided annotations need to reflect these changes immediately, or as fast as possible.\nclide-nlp uses the CoreNLP framework by the Stanford University’s NLP group that provides access to several NLP tools, including a coreference resolution system, which groups related entities in a text together, and a dependency parser, that describes the relations between sentence parts. Section 3.3 and Section 3.4 describe how we can build an NLP knowledge base based on these tools.\nChapter 4 shows how we can leverage the knowledge base to extract an ontology through a series of simple queries. The queries are described in detail in Section 4.1. Section 4.2 describes how the queries’ results are used to create an ontology. Section 4.3 discusses one possible way of how the ontology can be exported to an OWL8 ontology.\n6Natural Language Processing 7Groups of related entities in a text 8Web Ontology Language"
      }, {
        "heading" : "1. Introduction 9",
        "text" : "In Chapter 5 we describe an example application that uses the extracted ontology. We extend clide-nlp to create graphs from natural language specifications by leveraging the ontology. Figure 1.3 shows an example graph.\n10"
      }, {
        "heading" : "1. Introduction 11",
        "text" : "13\n2. Basics\nThis chapter introduces some of the concepts needed to understand this report.\nclide-nlp is implemented in Clojure. We first give a short overview of Clojure and its logic programming library core.logic. We conclude this chapter with a short introduction of CoreNLP, the NLP framework used by clide-nlp."
      }, {
        "heading" : "2.1. Clojure",
        "text" : "Clojure is a functional programming language and has several features that make it a good fit for NLP applications.\nClojure provides a REPL which allows us to patch in new code into a running system, enabling a small “thought-code-feedback loop” [7].\nIt provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).\nClojure is built on top of the JVM and has good interoperability support with Java libraries. This allows us to leverage all existing Java NLP libraries [6, 21].\nClojure’s macros can be used to extend the language when needed. This is used heavily by core.logic, which adds logic programming to Clojure.\nThe following table summarizes the aspects of Clojure’s syntax that is important for reading this report:\nType Example Description Function definition (defn f [x y] . . .) defn defines a function. Here we define the function f that takes 2 arguments x and y. Function call (f x) Clojure is a Lisp and uses prefix notation. Here we call the function f with argument x. Keyword :a Keywords are often used as keys in a map, because\nthey evaluate to themselves and can be used as functions that look themselves up in the associative data structures (e.g. a map) that is passed to them as their first argument.\nMap {:a \"Hello\" :b 0} A map with the key-value pairs (:a, \"Hello\") and (:b, 0) Vector [1 2 \"Hello\"] A vector with the elements 1, 2 and \"Hello\"\nThere is no special interoperability support for interfacing with Scala libraries, like e.g. Clide."
      }, {
        "heading" : "14 2.1. Clojure",
        "text" : "Interfacing with Scala directly from Java is already challenging, and interfacing with Scala from Clojure adds an additional complication. The work around is to write the components that use Clide directly in Scala and use Clojure’s runtime interface to call into Clojure code from Scala.\nMore information on Clojure is available in [6, 7]."
      }, {
        "heading" : "2.1.1. Logic programming with core.logic",
        "text" : "core.logic9 adds logic programming capabilities to Clojure. It is based on miniKanren, a logic programming library for Scheme, developed by William E. Byrd as part of his PhD thesis [2].\nBecause core.logic is a library for Clojure, we can mix functional and logic programming freely, dropping down to core.logic when we need it and use Clojure functional programming aspects otherwise [21].\nWe summarize the most important functions and macros of core.logic here:\nType Example Description Run a query (run∗ [q] . . .) Runs a core.logic query by trying to unify a\nvalue with q. Returns a list of all possible values for q\nCreate logic variables (fresh [a b] . . .) Creates two unbound logic variables a and b Unnamed logic variable (lvar) Returns a new logic variable Logical disjunction (conde\n[<branch1>] [<branch2>] . . .)\nTries all branches consecutively\nSoft cut (conda . . .) Like conde but stops the search as soon as a branch succeeds Feature extraction (featurec {:a 4 :b 5} {:a q}) Extracts features from maps. The example binds the logic variable q to 4. Unify (≡ q 4) Unifies the logic variable q with 4. Never unify (6≡ q 4) Adds a constraint to the logic variable q that it can never be 4. List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query\n(project [q] . . .) Extracts the value that is bound to the logic variable q. While in scope of a project q is a regular Clojure value and we can use regular Clojure functions with it.\nDomain constraint (in q (interval 1 10)) Makes sure that q is bound to a value in the interval [1, 10].\n9Available at https://github.com/clojure/core.logic"
      }, {
        "heading" : "2. Basics 15",
        "text" : "The presentation of the core.logic code shown here is based on the code presentation in [2, 8].\nPresentation Actual code <x>◦ <x>o A goal is written with a suffix o to distinguish it from\nalready defined functions on the functional programming side, while making clear that they have the same outcome in both paradigms [2]. E.g. cons◦ in core.logic and cons in Clojure\nrun∗ run* conde conde The branching macros have an added suffix to distinguish them from the built-in cond.conda conda ≡ == 6≡ !=\nThe Reasoned Schemer [8] provides a good introduction to miniKanren and in extension also core.logic.10\nExample. We create a new knowledge base and populate it with three facts about animals.\nRelation Kind Name animal cat Felix animal cat Mittens animal dog Waldo\nUsing this knowledge base, we can define a new goal that succeeds iff an animal is a cat or a dog. We make use of the predefined goal member◦ to check if the value of a logic variable is inside of a collection.\n1 (defn cat-or-dogo [name q] 2 (fresh [t] 3 (animal t name) 4 (member◦ t [\"cat\" \"dog\"]) 5 (≡ t q)))\nWe can then build a query to check what kind of an animal Waldo is:\n1 (run∗ [q] 2 (cat-or-dogo \"Waldo\" q)) 3 ⇒ (\"dog\")\nWaldo is a dog! And Benjamin?\n1 (run∗ [q] 2 (cat-or-dogo \"Benjamin\" q))\n10Their differences are described on core.logic’s Wiki available at https://github.com/clojure/core.logic/ wiki/Differences-from-The-Reasoned-Schemer.\n16 2.2. CoreNLP\n3 ⇒ ()\nBenjamin does not exist in the knowledge base, so the query returns no result. We can run the query in reverse to get all cats:\n1 (run∗ [q] 2 (cat-or-dogo q \"cat\")) 3 ⇒ (\"Felix\" \"Mittens\")\nWhile core.logic supports relational programming, our usage of several non-relation goals, like conda or project, makes all of our core.logic usage effectively non-relational."
      }, {
        "heading" : "2.1.2. Tawny-OWL",
        "text" : "Tawny-OWL11 is a Clojure library that provides a domain specific language for building OWL ontologies [13].\nclide-nlp uses Tawny-OWL for building OWL ontologies out of its custom ontologies it extracts from texts (see Section 4.3). Exporting OWL ontologies allows us to make use of the existing OWL tools, like e.g. querying them via SparQL [10]."
      }, {
        "heading" : "2.2. CoreNLP",
        "text" : "CoreNLP is an NLP framework that was created by the NLP group at Stanford University.12 It includes several components that facilitate developing NLP applications or algorithms. clide-nlp uses CoreNLP’s dependency parser and its coreference resolution system.\nThe dependency parser makes the underlying structures of sentences visible in the form of grammatical relations between sentence parts [4]. The output of this component can be modeled as a graph, where the grammatical relations are the graph’s edges and the nodes are the sentence parts. We call this graph semantic graph in this report. Examples of semantic graphs are available in Section 4.1 and Section 3.3. The grammatical relations are described in [5]. The dependency parser can collapse prepositions and coordinations into grammatical relations [4, 5]. clide-nlp uses this feature to simplify the resulting semantic graphs.\nThe dependency parser makes use of CoreNLP’s part-of-speech (POS) tagger. Its tagset is based on the POS tagset used by the Penn Treebank, described in [15]. Because semantic graphs contain POS tags and clide-nlp makes heavy use of semantic graphs, Table A.1 provides an overview over some of the tags provided by CoreNLP.\nThe deterministic coreference resolution system is used to identify entities refer to each other (also called mentions). It was introduced in [11, 12, 17, 18]. It competed in the CoNLL Shared Task 2011, where it achieved the highest score in both the closed and open tracks [12].\nSection 3.3.3 goes into more detail and shows an example coreference cluster.\n11Available at https://github.com/phillord/tawny-owl 12Available at http://nlp.stanford.edu/\n17\n3. Approach\nThis chapter introduces the components that constitute clide-nlp and how they interact with each other. We first describe how data flows between the components by giving a high-level architecture overview in Section 3.1. We then delve deeper into the implementation.\nclide-nlp receives a continuous stream of events (text operations, cursor movements, and annotation requests) from Clide that need to be integrated with clide-nlp’s underlying computation model. Section 3.2 describes this model and how we integrate changes into it.\nIn Section 3.3 we continue with a description of how CoreNLP is integrated into the system by revealing some pitfalls that occur when using the diverse data structures provided by CoreNLP and how we can avoid them. In Section 3.4 we then build a knowledge base from the data provided by CoreNLP which we can use with core.logic and that forms the basis for the remaining chapters.\nWe conclude this chapter with Section 3.5 and explain the user-facing side of clide-nlp by showing what annotations are provided and what caveats apply given the current annotation model of Clide."
      }, {
        "heading" : "3.1. Architecture",
        "text" : "Figure 3.1 shows how data flows between the components in clide-nlp.\nClide assistants need to provide a subclass of AssistantServer. AssistantServer has support for connecting and receiving messages from Clide built-in and abstracts away the underlying Akka implementation.\nclide-nlp calls its AssistantServer subclass AsyncAssistantServer. It receives events for file changes and cursor movements.\nAll file changes are passed to the reconciler, which uses referentially transparent functions. We pass them the current state of a file and it returns an updated version of that state. The AsyncAssistantServer instance is responsible for storing that state and retrieving it when needed.\nThe AsyncAssistantServer enqueues the reconciler state and the cursor position of the change in a queue that is used by the annotation-loop.\nThe annotation-loop reads one element (state and position) at a time from the queue and prepares Clide annotations based on it. See Section 3.5.3 for a list of provided annotations.\nThe annotations are not computed in the annotation-loop, but are sent to the assistant-loop,"
      }, {
        "heading" : "18 3.2. Reconciler",
        "text" : "which is responsible for realizing only those annotations that the users of clide-nlp want to see and then sending them to Clide.\nThe annotators that create Clide annotations make use of the reconciler’s lazy NLP annotations (see Section 3.2). The combination of lazy Clide annotations and lazy NLP annotations guarantees that clide-nlp only does work when it really has to."
      }, {
        "heading" : "3.2. Reconciler",
        "text" : "Clide is a collaborative editing environment. Multiple users may change the current file’s text at any time. As such clide-nlp needs a way to incorporate those text changes into its own data model.\nIn traditional IDEs the process which incorporates changes into its data models is called reconciliation. Clide itself does not provide built-in support for reconciling yet. clide-nlp has to provide that support itself."
      }, {
        "heading" : "3. Approach 19",
        "text" : "The reconciler has several related tasks, which are performed in the following order:\n1. Split the input text into separate chunks\n2. Replay Clide deltas to incorporate text changes and mark all chunks that have changed\n3. Compute NLP annotations for each changed chunk"
      }, {
        "heading" : "3.2.1. Chunking",
        "text" : "clide-nlp splits an input text into several parts to keep the time needed for (re-)computing the NLP annotations to a minimum and to make testing easier.\nIn the implementation provided with this report, clide-nlp splits an input text at the string \"\\n----\\n\".\nExample. The following text\nThe cat eats the mouse.\n----\nThe mouse is dead.\nis split into 3 chunks:\n1. The cat eats the mouse.\n2. ----\n3. The mouse is dead.\nEach chunk has an associated span. A span is the offset interval from the beginning of the text. In the example above, Chunk 1 has a span of [0, 25) and Chunk 2 is called a chunk separator.\nBecause a chunk is just a slice of an input text, more complicated chunkers are possible, and indeed would be more useful and realistic than the very simplistic chunker currently implemented.\nWe could e.g. treat comment blocks in a Java file as a chunk and the code in between blocks as chunk separators. CoreNLP might not understand tokens or characters used in Java comments and as a consequence, we would need to remove them first. If we replace them with whitespace before passing the comment to CoreNLP, we make sure that we can map to the original text in an easy way by mirroring the spans inside the original text and inside the replacement text.13\n13CoreNLP does something similar in its cleanxml annotator to remove XML tags from an input text before parsing it."
      }, {
        "heading" : "20 3.2. Reconciler",
        "text" : "Example. The Java comment\n/** * This is a comment */\ncan be written as S = \"/∗∗\\n␣∗␣This␣is␣a␣comment\\n␣∗/\"\nThe substring T = \"This␣is␣a␣comment\"\nhas the span [7, 24) in S. If we replace all special characters not understood by CoreNLP with spaces, we would get the string\nS′ = \"␣␣␣\\n␣This␣is␣a␣comment\\n␣␣␣\"\nThe span of T in S′ is still [7, 24)."
      }, {
        "heading" : "3.2.2. Incorporating text changes",
        "text" : "In Clide text changes are described as a list of operations that describe the steps needed to transform an old version of a text into a new version. There are three operations [20]:\n• Retain(n)\n• Insert(s)\n• Delete(n)\nSince Clide is written in Scala Retain, Insert and Delete are implemented using Scala case classes.14 While it is possible to work with theses classes in Clojure, it is easier to translate them to use Clojure’s data structures instead.\nThe translation is straightforward. For each operation, replace\nRetain(n) with [:retain n] Insert(s) with [:insert s] Delete(n) with [:delete n]\nExample. The operations [Retain(5), Insert(\"hallo\"), Retain(15)] are translated into the Clojure data [[:retain 5] [:insert \"hallo\"] [:retain 15]].\nTo apply the operations we need to maintain a cursor position starting at 0. The cursor’s position is the position in the original text, not the edited text. The original text is immutable and is not changed. The edited text is built by applying each operation sequentially:\n• [:retain n] moves the cursor n characters ahead and inserts them in the edited text.\n14Case classes are algebraic data type constructors and allow pattern matching."
      }, {
        "heading" : "3. Approach 21",
        "text" : "• [:insert s] inserts the string s at the current cursor position in the edited text. This will not move the cursor, because it is relative to the original text.\n• [:delete n] deletes the next n characters at the current cursor position and moves the cursor n characters ahead.\nExample. Given the input text \"This␣is␣a␣test.\" and the operations [[:retain 9] [:insert \"the\"] [:delete 1] [:retain 6]], we get the text \"This␣is␣the␣test.\" after applying them.\nAs the reconciler splits a text into several chunks there are some more concerns to address.\nThe changes to a text can be\n1. inside a chunk separator, i.e. between two chunks A and B15 in which case there are two possible scenarios:\na. A remains unchanged and the spans of B and of all chunks that follow it have to be updated.\nb. A and B have to be merged together because the chunk separator is not valid anymore. This would also change the spans and indices of all chunks that follow A and B.\n2. inside one chunk C, in which case C’s span (end offset) and the span of all chunks that follow C must be updated to reflect the changes.\nIn the actual reconciler implementation changes between two chunks are detected after applying the edit operations and rechunking the text. If the number of chunks changed, the reconciler is simply reinitialized. This greatly simplifies the implementation, but all NLP annotations are lost and need to be rebuilt. An ideal implementation would have to follow all scenarios above."
      }, {
        "heading" : "3.2.3. Chunk annotations",
        "text" : "Each chunk has associated annotations that are updated when a chunk changes. The annotations are added to a Clojure map with lazy evaluation semantics. This model allows the reconciler to remain fast even when there are continuous changes. The chunk annotations are only realized, and thus computed, outside of the reconciler.\nThe graph in Figure 3.2 shows how the annotations depend on each other. Note that graph is the central nexus of clide-nlp that pulls all of its parts together. As such every aspect of clide-nlp is mirrored in it.\nThe annotations have the following meaning:\n:text, :corenlp-pipeline The graph’s inputs are a previously constructed CoreNLP pipeline16\nand the chunk’s text.\n:corenlp-annotation A CoreNLP annotation is created based on the input text. The annotation provides access to all primitive NLP constructs we need (which includes the coreference chain, the semantic graphs, all split sentences, and information about each token).\n15Chunk B follows chunk A. 16The pipeline needs to be setup to use CoreNLP’s depedency parser and its coreference resolution system."
      }, {
        "heading" : "22 3.2. Reconciler",
        "text" : ":semantic-graphs Extracts all semantic graphs from the annotation and creates a Clojure representations of them. In CoreNLP the class that represents nodes in a semantic graph is called IndexedWord. All IndexedWord instances are mapped to a word map for later use (see Section 3.3.2).\n:coref-chain-map Extracts all coreference clusters from the CoreNLP annotation (see Section 3.3.3).\n:sentences Extracts information about each of the input text’s sentences from the annotation and builds a list of sentence maps (see Section 3.3.1).\n:knowledge-base Builds a knowledge base for use with core.logic out of the coref chain map and semantic graphs. The process is described in Section 3.4.\n:triples Runs triple builders on the knowledge base that extract useful information gathered from the semantic graphs and builds a list of triples. The triple’s subject, predicate, and object only refers to a node in the semantic graph of one sentence. See Chapter 4 for more details.\n:grouped-triples Runs triple builders on the knowledge base and groups the triples’ subjects"
      }, {
        "heading" : "3. Approach 23",
        "text" : "and objects by their coreference cluster. A group is a list of coreferent or otherwise related words. In contrast with the triples extracted by :triples, the grouped triple’s subject and object are a list of related words that can span multiple sentences (the whole text) instead of only one sentence. See Section 4.2 for more details.\n:reified-triples Multiple grouped triples can all have the same subject or object groups. :reified-triples assigns a unique name to each group i.e. the groups are made real (reified) by giving them a name (see Section 4.2).\n:reified-triples-knowledge-base Adds the reified triples to the knowledge base for use by applications that don’t want to search the triples sequentially.\n:draw An annotation that is introduced in detail in Chapter 5. It tries to draw a graph specified in the input text and to warn about simple ambiguous sentences.\n:ontology Builds an OWL ontology (see Section 4.3).\nAn annotation is only realized when it is directly needed or used by a dependent annotation. As a result, clide-nlp does not always have to compute all annotations if the text changed and only does work if it is really needed.\nExample. If a client accesses the :knowledge-base annotation only the following annotations are realized: :semantic-graphs, :coref-chain-map, :corenlp-annotation, :text.17"
      }, {
        "heading" : "3.3. Integration of CoreNLP",
        "text" : "clide-nlp tries to not rely on CoreNLP’s data structures, because the data structures need to participate in core.logic unification. Due to the mutable nature of CoreNLP’s data structures extending them to reliably support unification is problematic.\nAdditionally, there are inconsistencies in the usages of 0- or 1-based indices in CoreNLP’s data structures. This is corrected when constructing clide-nlp data structures and allows for easier matching up of the different data structures based on sentence and token indices.\nAll data structures are implemented using Clojure records. Records are reified maps, which compile down to Java classes. They implement the correct interfaces, so that they can be treated as maps, which we will do from this point on.\nWe introduce each record by\n• giving a short description of its use in clide-nlp,\n• by listing its available keys with a description of the content of their values,\n• and by showing examples with actual data.\n17Note that :text is the input to the graph and as such always realized."
      }, {
        "heading" : "24 3.3. Integration of CoreNLP",
        "text" : ""
      }, {
        "heading" : "3.3.1. Accessing sentences of a text",
        "text" : "While accessing individual sentences of a text is not important for the core task of clide-nlp (extracting an ontology from a text), we need them to create Clide annotations that refer to a whole sentence (see Section 3.5).\nSentence maps have the following keys:\n:index The index of the sentence starting at 0. :span The 0-based character index span [a, b) of the sentence. The sentence starts at offset a and goes up to offset b. :text The text of the sentence.\nExample. The text \"Felix is a cat. Waldo is a dog. Tweety is a bird.\" results in the following sentence maps:18\n:index 0 :span [0, 15) :text Felix is a cat.\n:index 1 :span [16, 31) :text Waldo is a dog.\n:index 2 :span [32, 49) :text Tweety is a bird."
      }, {
        "heading" : "3.3.2. Word maps",
        "text" : "Semantic graph nodes are an integral part of the triple builders introduced in Chapter 4.\nThe semantic graph node class in CoreNLP is called IndexedWord. The information that an IndexedWord object provides, is used to create a word map with the following keys:\n:sentence The sentence index this word map refers to. This matches the sentence maps’ :index value. :index The index of the word’s token starting at 1. CoreNLP consistently starts at 1 when counting tokens. :span The 0-based character index span [a, b) of the word map. :tag The word’s part-of-speech tag. :lemma The word’s lemma. :token The word’s token.\n18When checking the spans do not forget to include the spaces between the sentences!"
      }, {
        "heading" : "3. Approach 25",
        "text" : "Example. The semantic graph for the input text \"Felix is a cat.\" has the following word maps:\ncat NN\nFelix NNP a DTis VBZ\nnsubj det\ncop\n:sentence 0 :index 1 :span [0, 5) :tag NNP :lemma Felix :token Felix\n:sentence 0 :index 2 :span [6, 8) :tag VBZ :lemma be :token is\n:sentence 0 :index 3 :span [9, 10) :tag DT :lemma a :token a\n:sentence 0 :index 4 :span [11, 14) :tag NN :lemma cat :token cat"
      }, {
        "heading" : "3.3.3. Coreferences",
        "text" : "clide-nlp uses CoreNLP’s coreference resolution system to identify which entities in a text are similar to other entities in a text. Coreferences are grouped in clusters. A cluster is made up of mentions and we map them to mention maps with the following keys:\n:cluster-id The coreference cluster id the mention map is a part of. :sentence The index of the sentence that contains this mention map. The indices in\nCoreNLP start at 1 here. We correct them to be 0-based, and as a consequence match a sentence map’s :index and a word map’s :sentence value.\n:index-span The index span [a, b) refers to the tokens starting at the word map with index a and ends before the word map with index b. The indices are 1-based again, but we do not need to adjust them here, because the token indices of word maps are 1-based, too. :text A clear text representation of the mention map.\nCoreNLP’s coreference system provides additional information about each mention. Information about a mention’s gender, its animacy or its number (plural or singular), is however currently unused by clide-nlp.\nExample. The text \"Felix is a cat.\" has the following coreference cluster and associated mention maps:\nFelix [0:1-2]\na cat [0:3-5]\n:cluster-id 1 :sentence 0 :index-span [1, 2) :text Felix\n:cluster-id 1 :sentence 0 :index-span [3, 5) :text a cat\nWe see that the coreference resolution system in CoreNLP identified Felix to have the same meaning as a cat."
      }, {
        "heading" : "26 3.4. Building an NLP knowledge base",
        "text" : ""
      }, {
        "heading" : "3.4. Building an NLP knowledge base",
        "text" : "This section describes how the data structures introduced in Section 3.3 are inserted into a core.logic database.\nChapter 4 makes extensive use of the knowledge base and provides usage examples.\nWe first need to define the relations that we want to provide. They are described in further detail later. clide-nlp’s knowledge base provides the following relations:\n(word-map w) provides access to word maps (semantic graph nodes). (depends dep reln gov) provides access to semantic graph edges. (same-as w1 w2) succeeds iff the word maps w1 and w2 can be treated as referring to\nthe same word.\nNext we need to insert facts into the knowledge base. Given an input text,\n1. for every word map w of a semantic graph of a sentence of the text, insert the fact (word-map w).\n2. for every semantic graph edge e = (dep, reln, gov) of a semantic graph of a sentence of the text, insert the fact (depends dep reln gov).\n3. for the word maps w1 and w2 and using word-map, depends and coreference cluster mentions, determine if w1 and w2 can be treated as referring to the same word, then insert the facts (same-as w1 w2) and (same-as w2 w1).\n(word-map w)\nword-map provides access to the word map w. It is a unary relation. To unify with information from a word map, it needs to be used in conjunction with core.logic’s feature extraction goal featurec (see Section 2.1.1).\nExample. Using core.logic’s featurec goal, we can limit a query to only succeed with maps with specific features. The following query returns all word maps which have a tag NN and an index 0.\n1 (run∗ [q] 2 (word-map q) 3 (featurec q {:tag \"NN\" :index 0}))\nBecause matching a specific set of part of speech tags is used heavily by the triple builders introduced in Chapter 4, the following helper goals are defined:"
      }, {
        "heading" : "3. Approach 27",
        "text" : "(tag◦ w tags) suceeds iff w has one of the tags in the vector tags. (verb◦ w) succeeds iff w is a verb, i.e. if it has one of the tags VB, VBD, VBG, VBN, VBP or VBZ. (noun◦ w) succeeds iff w is a noun or pronoun, i.e. if it has one of the tags NNP, NN, NNS. PRP or PRP$. (wh-word◦ w) succeeds iff w is a wh-word19 i.e. if it has one of the tags WDT, WP,\nWP$ or WRB.\ntag◦ is the basis of the implementations of all of these goals. tag◦ can be defined in the following way:\n1 (defn tag◦ 2 [w tags] 3 (fresh [tag] 4 (word-map w) 5 (featurec w {:tag tag}) 6 (member◦ tag tags)))\n(depends dependent relation governor)\nWhile word-map provides access to semantic graph nodes, depends provides access to semantic graph edges.\nAn edge goes from the word map governor to the word map dependent with the grammatical relation relation. We insert every edge of every semantic graph of every sentence of a text into the knowledge base.\nrelation can be either a string containing a typed dependency relation (see [5]) or if the relation is a collapsed relation, a vector of the first and second part of the relation (e.g. if the relation in the semantic graph was prep_of, it gets split into the vector [\"prep\", \"of\"]). Splitting a collapsed dependency in that way keeps core.logic queries simple by making use of its native support for unifying vectors. This allows searching for specific prepositions or all prepositions in a simple manner.\nExample. The following query returns the governor and dependent word maps of all edges with a prepositional relation (e.g. prep_of or prep_in) from every semantic graph of a text.\n1 (run∗ [q] 2 (fresh [dep gov p] 3 (depends dep [\"prep\" p] gov) 4 (≡ q [dep gov])))\n(same-as w1 w2)\nsame-as asserts that the word map w1 is the same as the word map w2 and that we can treat"
      }, {
        "heading" : "28 3.4. Building an NLP knowledge base",
        "text" : "the words as being instances of the same word group.\nsame-as should be commutative, so that the order of w1 or w2 does not matter. If (same-as w1 w2) succeeds, (same-as w2 w1) succeeds, too.\nBecause w1 and w2 are instances of the same word group and we like the word groups to be about a concrete thing, we want to limited them to only include pronouns, nouns, or determiners. Including adjectives e.g. does not make sense because they are properties of word groups. Verbs are used between two or more word groups.\nsame-as is important for grouping triples (see Section 4.2). There are several aspects for when we can consider two word maps the same.\nTo be included in the same-as relation, the word maps w1 and w2 need to fulfill at least one of the following rules:\n• w1 and w2 need to map to the same coreference cluster. We need to find the corresponding word maps of each of the cluster’s mentions. We can do this in a core.logic query by constraining a word maps index to be inside of the mention’s index span:\n1 (let [[start end] (:index-span mention), sentence (:sentence mention)] 2 (run∗ [q] 3 (fresh [index tag] 4 (word-map q) 5 (featurec q {:index index, :tag tag, :sentence sentence}) 6 (in index (interval start (dec end))))))\nWe further limit the query result to only include word maps that represent pronouns, nouns, or determiners.\nWe run the query for every cluster mention and select every 2 combination of the found word maps and record the facts:20\n(same-as w1 w2) (same-as w2 w1) (same-as w1 w1) (same-as w2 w2)\nWe repeat this process for every coreference cluster.\n• The query\n1 (run∗ [w1 w2] 2 (noun◦ w1) 3 (noun◦ w2) 4 (depends w1 \"nn\" w2))\nsucceeds for w1 and w2.\nnn is the noun compound modifier dependency relation that asserts that one noun modifies another noun [5].\n20While facts might be recorded twice, we can safely ignore this."
      }, {
        "heading" : "3. Approach 29",
        "text" : "By including compound nouns in same-as, we ensure that every word of a compound noun is assigned to the same word group later (see Section 4.2).\n• w1 and w2 need to be linked by a wh-word. Word maps that are linked by a wh-word can be found with the following query:\n1 (run∗ [w1 w2] 2 (fresh [w] 3 (wh-word◦ w1) 4 (depends w1 \"nsubj\" w) 5 (depends w (lvar) w2)))\nCoreNLP’s coreference system does not include wh-words in its mentions. Some triples found by the triple builders in Chapter 4 have a wh-word as their subject and we need to make sure that they can be grouped together with the other groups and do not create a group by themselves."
      }, {
        "heading" : "3.5. Clide annotations",
        "text" : "Clide annotations are used to provide rich information about specific parts of a text. They are currently static and non-interactive21. There is e.g. no way to jump to a specific word in the document from an annotation. While this limits their usefulness, it does not prevent them from being helpful.\nThey follow the same model as the operations sent by Clide (see Section 3.2.2) and are represented as a list of annotation operations, called an annotation stream. In Clide they are simply called annotations, but we use the term annotation stream to distinguish them from the annotation lists they contain."
      }, {
        "heading" : "3.5.1. Annotation streams",
        "text" : "An annotation stream is a list of annotation operations. There are two types of operations:\n• Plain(n)\n• Annotate(n, annotations)\nwhere n is the annotation’s length and annotations is a list of tuples (type, content) with type being the annotation type and content a string containing the actual annotation content.\nWe again translate the Scala syntax into Clojure data and replace\nPlain(n) with [:plain n] Annotate(n, annotations) with [:annotate n annotations]\nTo apply an annotation stream we need to maintain a cursor position starting at 0. The annotation is applied by applying each action sequentially and mutating the cursor position afterwards. 21This means that you cannot interact with the annotation itself, because they are view-only. You can however\nrequest a new annotation.\n30 3.5. Clide annotations\n• [:plain n] skips n characters from the current cursor position ci, and adds n to it: ci+1 = ci + n.\n• [:annotate n annotations] moves the cursor n characters ahead and annotates the text span from [ci, ci+1) where ci+1 = ci + n. Clide then interprets the annotation list annotations and displays them.\nwhere i the index of the operation in the list\nc0 0 ci the cursor position before applying the operation\nci+1 the cursor position after applying the operation\nThere is a direct correspondence between annotation and edit operations (see Section 3.2). Ignoring annotations, we can treat :plain and :annotate as :retain operations [20].\nAn annotation stream should span the whole text, that is by summing up the n-s of each operation, we would get the text length.\nClide defines several annotation types. clide-nlp uses the following subset of them:\nClass sets the CSS class to use for the annotation.\nTooltip sets the tooltip used when hovering over the annotation.\nWarningMessage display a warning inline.\nOutput for displaying generated information, that is not displayed inline by default.\nClide allows HTML inside of its annotations, which means that we can display richer annotations than only simple plain text annotations.\nExample. Given the text \"The␣cat␣is␣hungry.\" of length 18 and the annotation stream [[:plain 4], [:annotate 3 [[:Class \"error\"]]], [:plain 11]] and assuming [:Class \"error\"] is meant to color the annotated text red, applying the annotation stream to the text results in \"The␣cat␣is␣hungry.\""
      }, {
        "heading" : "3.5.2. Annotation levels",
        "text" : "While Clide annotations can annotate arbitrary ranges of a text, it is useful to distinguish between different annotation levels in clide-nlp:\nText Annotates the whole text\nChunk Annotates a chunk (as defined in Section 3.2)\nWord Annotates a single word\nSentence Annotates a whole sentence\nEvery annotation in clide-nlp, with the exception of a text level annotation, is relative to a chunk. By doing so, we keep the annotation creation as simple as possible."
      }, {
        "heading" : "3. Approach 31",
        "text" : "As discussed in Section 3.2, a chunk has an associated span that indicates the chunk’s text position inside of the global text. A chunk’s internal span begins at 0. CoreNLP never sees the whole text at once, but instead only sees the texts of every chunk separately, so all spans returned by CoreNLP also begin at 0.22 It follows that the NLP knowledge base is chunk local, too. As the annotations created by clide-nlp all use CoreNLP annotations or the NLP knowledge base, we ideally should use chunk local offsets when creating annotation streams. We later project their chunk local offsets to offsets that Clide can interpret correctly."
      }, {
        "heading" : "3.5.3. Annotations provided by clide-nlp",
        "text" : "In this section we describe each annotation that clide-nlp provides by showing an example of how they appear in Clide. The annotations make use of one more of the chunk annotations as described in Section 3.2. There might be some overlap in their names, but they should be treated as separate entities. The names are presented to the user by Clide, who can enable them individually (see the names to the left with the “eyes” in Figure 1.1).\nchunk-separators Level: Text\nHighlights the chunk separators. Chunks are described in Section 3.2. This makes the chunkers decisions visible.\nsemantic-graph Level: Sentence\nHighlights a sentence and displays the sentence’s semantic graph. This annotation is highly useful when creating a new triple builder.\n22After some correction (see Section 3.3)\n32 3.5. Clide annotations\ncoref-cluster Level: Chunk\nShows the coref clusters that CoreNLP found for the selected chunk’s text.\nsame-as Level: Word\nA direct interface to the knowledge base that uses the same-as relation to highlight the related words of the selected word.\ntriples Level: Chunk\nShows the raw triples extracted from running the triple builders on the selected chunk’s text.\ngrouped-triples Level: Chunk\nShows the triples’ groups.\n3. Approach 33\nreified-triples Level: Chunk\nShows a table with all reified triples extracted from the selected chunk’s text.\nreified-name Level: Word\nThis annotation is identical to the same-as annotation, but additionally adds a tooltip to the highlighted words, showing the word group they belong to.\n35\n4. Triples\nIn the previous chapter we built an NLP knowledge base. This chapter introduces triple builders that runs simple core.logic queries on it and extracts triples. Triples have the form (subject, predicate, object). They represent a unit of useful information.\nWhere possible a triple’s subject and object each have one corresponding word map in a semantic graph. Using the knowledge base introduced in the previous chapter, we group the triple’s subject and object with other related subjects or objects of other triples, yielding an ontology."
      }, {
        "heading" : "4.1. Triple builders",
        "text" : "Triple builders extract triples. Triples have the form (subject, predicate, object). They represent a unit of useful information that is directly extracted from the semantic graph.\nA subject usually does something (predicate) with an object. Every subject and object is directly linked to a word map. For predicates however this is not always possible, because there are predicates which are implied by the semantic graph (see the triple builders nsubj-amod or possessive for an example of this). These predicates are called derived and written with a colon prefix (e.g. :be) to distinguish them from word map predicates.\nWhile the triple builders only extracts information from semantic graphs, they could be extended to include information from ontologies like WordNet [16], VerbOcean [3] or DBpedia [1] to further constrain the triples that are found.\nA triple builder is implemented as a core.logic goal.23 It either succeeds with a triple or it does not. All triple builders are tied together in a logical disjunction (conde) and are all tried in turn. Most of the triple builder’s names are directly derived from the semantic graph relation’s names they use.\nWe show each triple builder with\n• the actual core.logic query,\n• an example input sentence,\n• the corresponding semantic graph for the input sentence, with highlights for the relevant nodes and edges to make it easier to follow along,\n• a table of the triples that it found when run on the input sentence, displaying the lemmas of its subject, predicate or object,\n• and a discussion of the triple builder. 23Note that we deviate from the core.logic convention of marking a goal with a postfix o here."
      }, {
        "heading" : "36 4.1. Triple builders",
        "text" : "Note that a triple builder may succeed multiple times and thus may find more than one triple. nsubj-amod\ncat NN\neats VBZ\nhungry JJThe DT\nnsubj\ndet amod\nInput text The hungry cat eats.\nQuery\n(fresh [subj adj] (noun◦ subj) (depends adj \"amod\" subj) (≡ triple [subj :be adj]))\nTriples found\ncat :be hungry\nThe meaning of a subject can be modified with an adjectival modifier (amod) [5]. This triple builder captures the fact that cat refers not just to cat but to a hungry cat. Because there is no node in the graph that can take the role of a predicate, we introduce a derived predicate :be instead.\nnsubj-pred-dobj\nthe DT\nIt PRP mouse NN\neats VBZ\nnsubj dobj\ndet\nInput text It eats the mouse.\nQuery\n(fresh [subject activity object] (depends subject \"nsubj\" activity) (depends object \"dobj\" activity) (conde\n[(≡ triple [subject activity object])] [(fresh [subject2]\n(depends subject2 [\"conj\" \"and\"] subject) (≡ triple [subject2 activity object]))]))\nTriples found\nit eat mouse\nThis triple builder captures facts from one of the simplest kind of sentences. A subject directly connected to an object with a predicate."
      }, {
        "heading" : "4. Triples 37",
        "text" : "nsubj-VB\neats VBZ\ncat NN\nThe DT\ndet\nnsubj\nInput text The cat eats.\nQuery\n(fresh [subj vb] (noun◦ subj) (tag◦ vb [\"VBZ\" \"VBD\" \"VBP\"]) (depends subj \"nsubj\" vb) (≡ triple [subj :be vb]))\nTriples found\ncat :be eat\nSome sentences have intransitive verbs24 and no object. However, they might still contain useful information. Here we capture the fact that the cat eats. To do this, the verb becomes our triple object and we use a derived predicate :be. This mirrors the behavior used in e.g. the triple builder nsubj-amod for adjectives. We limit ourselves to sentence with verbs that are in the past tense (tag VBD) or singular present (tags VBP and VBZ) to not always trigger this triple builder for all sentences with an nsubj relation between a noun and a verb.\nnsubj-adj-cop\nis VBZ often RBcat NN\nThe DT\nfull JJ\nnsubj advmod\ndet\ncop\nInput text The cat is often full.\nQuery\n(fresh [subject adj cop] (depends subject \"nsubj\" adj) (depends cop \"cop\" adj) (≡ triple [subject cop adj]))\nTriples found\ncat be full\nThis triple builder captures adjectives of subjects, e.g. that the cat is full. We can qualify the object full with an additional triple. See the next triple builder nsubj-advmod.\n24verbs with no object"
      }, {
        "heading" : "38 4.1. Triple builders",
        "text" : "nsubj-advmod\noften RBis VBZ\nThe DT\ncat NN\nfull JJ\nnsubj\ndet\nadvmod\ncop\nInput text The cat is often full.\nQuery\n(fresh [subject predicate advmod cop] (depends subject \"nsubj\" predicate) (depends advmod \"advmod\" predicate) (conda\n[(verb◦ predicate) (≡ triple [subject predicate advmod])] [(≡ triple [predicate :be advmod])]))\nTriples found\nfull :be often\nIn combinations with nsubj-adj-cop finds additional descriptions of an adjective, but also additional properties of subjects. Following e.g. the triple builders nsubj-amod or nsubj-VB, :be is used as our predicate again. The object from the triple found in the example of nsubj-adj-cop full is qualified here with often.\nnsubj-pred-acomp\nlooks VBZ\nIt PRP hungry JJ\nnsubj acomp\nInput text It looks hungry.\nQuery\n(fresh [subject activity acomp] (depends subject \"nsubj\" activity) (depends acomp \"acomp\" activity) (≡ triple [subject activity acomp]))\nTriples found\nit look hungry\nVerbs and adjectives have an acomp (adjectival complement) relation, if an adjective can be treated as the verb’s object [5]. The triple extraction is straightforward. The adjective becomes the triple’s object. The verb’s subject the triple’s subject and the verb itself the predicate."
      }, {
        "heading" : "4. Triples 39",
        "text" : "nsubj-pred-xcomp\nenter VB\nto TO\nmanaged VBD\nHe PRP\nthe DT\nhouse NN\nnsubj xcomp\ndet\naux dobj\nInput text He managed to enter the house.\nQuery\n(fresh [subject activity xcomp object relation] (depends subject \"nsubj\" activity) (depends xcomp \"xcomp\" activity) (depends object relation xcomp) (member◦ relation [\"advmod\" \"dobj\"]) (conde\n[(≡ triple [subject activity xcomp])] [(≡ triple [subject xcomp object])]))\nTriples found he manage enter he enter house\nenter is an open clausal complement (xcomp) of managed. enter does not have its own subject but refers to the subject of managed (He) [5]. There are two useful triples that can be extracted from the graph. One is the fact that He managed to do something (enter) and one is a fact about what He entered (the house).\nnsubjpass-pred-agent\na DT\nwarning NNis VBZ\nswayed VBN\nHe PRP\nnsubjpass agent\ndet\nauxpass\nInput text He is swayed by a warning.\nQuery\n(fresh [subject predicate object] (depends subject \"nsubjpass\" predicate) (depends object \"agent\" predicate) (≡ triple [object predicate subject]))\nTriples found\nwarning sway he\nAn agent is “introduced by the preposition by” [5]. This also implies a passive subject (he), which we will use for our triple’s object. The agent (warning) is used as the triple’s subject, because it does something with the passive subject."
      }, {
        "heading" : "40 4.1. Triple builders",
        "text" : "agent-ccomp-dobj\nyou PRPY NN\ndo VBP\nthat IN\nis VBZX NN\nknown VBN\nIt PRP\nnsubj\nccomp\nmark\nagent auxpass nsubjpass\ndobj\nInput text It is known by X that you do Y\nQuery\n(fresh [agent object predicate ccomp] (depends agent \"agent\" predicate) (depends ccomp \"ccomp\" predicate) (depends object \"dobj\" ccomp) (≡ triple [agent :about object]))\nTriples found\nx :about y\nThis triple builder extracts the fact that X talks about Y. Because there is no direct graph node we could use for the predicate here, we introduce another derived predicate :about. Any other information from this sentence, will be captured by other triple builders.\npossessive\nis VBZ\nred JJ\nhouse NN\nJohn NNP\nnsubj\nposs\ncop\nInput text John’s house is red.\nQuery\n(fresh [subject object] (depends subject \"poss\" object) (noun◦ subject) (noun◦ object) (≡ triple [subject :have object]))\nTriples found\nJohn :have house\nWe want to determine what kind of possessions a subject has. The semantic graph has the relation poss (possession modifier) for this. Because we are missing a direct predicate in the graph, we introduce a derived predicate :have."
      }, {
        "heading" : "4. Triples 41",
        "text" : "nsubjpass-ccomp\nC NN are VBP\nB NN\nas RB\nX NN\nY NN\nare VBP\nconnected VBN\nadvmod\nconj_andnsubjpass ccomp\nauxpass\nnsubjpass\ncop\nccomp\nconj_and\nInput text X and Y are connected as are B and C\nQuery\n(fresh [subj1 predicate subj2 reln1 reln2] (depends subj1 reln1 predicate) (depends subj2 reln2 predicate) (member◦ reln1 [\"nsubjpass\" \"ccomp\"]) (member◦ reln2 [\"nsubjpass\" \"ccomp\"]) (depends subj1 [\"conj\" \"and\"] subj2) (conde\n[(≡ triple [subj1 predicate subj2])] [(≡ triple [subj2 predicate subj1])]))\nTriples found c connect b b connect c y connect x x connect y\nThe query searches for nouns that are connected via some predicate and have a clausal complement (ccomp) or passive nominal subject (nsubjpass) relation with it. Because the subject/object-order of the nouns in the resulting triple shouldn’t matter, the query is allowed to succeed for all possible combinations of them, with the additional constraint that the nouns have to be connected via some conjunction. This prevents extracting wrong triples, like e.g. “b connect x”.\nCounterexample. This triple builder sometimes finds information that is obviously wrong. If we vary the sentence a little bit by changing “connected” to “proven”, we get essentially the same semantic graph and triples with wrong information. Running the triple builder on the sentence „X and Y are proven as are B and C” would return the triples y prove x x prove y c prove b b prove c\nClearly this is wrong, “proven” is an intransitive verb here and X and Y did not “prove” each other, but were proven by some (in this case unknown) agent. There is something missing here. Integrating a verb ontology might help to determine if a verb is transitive or intransitive. In the case of intransitive verbs a more appropriate result (matching the triples found by the triple builder nsubj-VB) would be: x :be prove y :be prove c :be prove b :be prove\n42 4.1. Triple builders\nprep-noun\n2 CD\ncm NN\nThey PRP\nhave VBP\na DT\ndistance NN\nnum\nnsubj\nprep_of det\ndobj\nInput text They have a distance of 2 cm.\nQuery\n(fresh [obj subj activity prep] (depends obj [\"prep\" prep] subj) (noun◦ obj) (noun◦ subj) (project [prep]\n(≡ triple [subj (keyword prep) obj])))\nTriples found\ndistance :of cm\nThis triple builder captures preposition between two nouns (or pronouns). Because CoreNLP collapses prepositions, we introduce derived predicates for each preposition. In this case because of the edge between the subject and object (prep_of) the predicate will be :of.\nnoun-prep-noun\nY NNsame JJthe DT\nstate NNX NN\nis VBZ\nnsubj prep_in\nprep_asdet amod\nInput text X is in the same state as Y\nQuery\n(fresh [obj subj prep activity] (depends subj \"nsubj\" activity) (depends obj [\"prep\" prep] activity) (noun◦ obj) (noun◦ subj) (verb◦ activity) (conde\n[(project [prep] (≡ triple [subj (keyword prep) obj]))] [(≡ triple [subj activity obj])]))\nTriples found x be state x :in state\nThe complement to prep-noun that captures preposition that are indirectly connected to a noun. Like in prep-noun we again use a derived predicate for capturing the preposition. Because the nouns are not connected directly, we let the query succeed twice. Once for capturing the preposition as our predicate (:in) and once for capturing the actual predicate from the graph (is).\n4. Triples 43\nnoun-num\n5 CD\nhas VBZ\napples NNSHe PRP\nnsubj dobj\nnum\nInput text He has 5 apples.\nQuery\n(fresh [unit num] (depends num \"num\" unit) (noun◦ unit) (tag◦ num [\"CD\"]) (≡ triple [unit :be num]))\nTriples found\napple :be 5\nThis triple builder captures numeric modifiers (num) [5] of nouns, e.g. how many of apples there are (5 ). Because we are missing a direct predicate in the graph, we will use the derived predicate :be.\nadvmod-npadvmod-num\nA DT\nwide JJ\n10 CD\ncm NNis VBZ\nnsubj npadvmod\nnum\ncop\nInput text A is 10 cm wide.\nQuery\n(fresh [advmod unit num] (conde\n[(depends advmod \"advmod\" (lvar))] [(word-map advmod) (featurec advmod {:tag \"JJ\"})])\n(depends unit \"npadvmod\" advmod) (depends num \"num\" unit) (≡ triple [advmod :be unit]))\nTriples found\nwide :be cm\nThe complement to noun-num to capture the unit of a measurement.\nTable 4.1 shows all triples that are captured by running them on an example text that is used in Chapter 5 to draw a graph from the text.\nLooking at the table it is clear that all necessary information for this is available, however we currently do not have any way of discerning if an instance of e.g. “distance” is the same as another instance of “distance” or what “it” refers to."
      }, {
        "heading" : "44 4.2. Reifying triples",
        "text" : ""
      }, {
        "heading" : "4.2. Reifying triples",
        "text" : "Currently the triples’ subjects or objects refer to one word map only. For example, looking at Table 4.1 we see several subjects with a “Test” or “it” lemma. There is no way that we can know if the instances refer to the same entity or to distinct entities.\nBy using the same-as relation we build in Section 3.4, we can group the triples’ subjects and objects with all other words in the knowledge base that can be treated as refering to the same entity, reifying the triples.\nIf we apply same-as to every subject and object of our triples our table might look like Table 4.2, where the subjects and objects are replaced by the word groups that the original word belonged to.\nExample. Let us look at the coreference clusters found by CoreNLP for our input text. This gives us an approximation of what the same-as relation looks like:\nNode Noname [2:1-3]\nnode End [3:4-6]\nNode Start [3:1-3]\nnode Start [3:10-12]\nnode End [0:5-7]\nEdge Test [4:1-3]\nEdge Test [0:1-3]\nnode Start [2:4-6]\nnode Start [0:10-12] node Noname [3:13-15]\nIt [1:1-2]\nIf we look at the subject word map w of the triple Test go End And we search for every v for which (same-as w v) holds, we get the word group:25 Edge (0,1), Test (0,2), it (1,1), Edge (4,1)\nBecause working with a list of word maps is cumbersome and also hard to refer to, we give each word group a unique name. A word group’s name is made up of the nouns (this excludes Wh-words or pronouns) of each word in the group. Because some names may not be unique, we add a number to it. We increment the number every time there is a duplicate name for a group.\nA predicate can be derived or refer to a concrete word map. We simply reify the predicates by making them all derived. For non-derived predicates we use it’s lemma and derived predicates are copied verbatim. This essentially also groups the predicates together.\nThe result of applying these steps to Table 4.2 can be seen in Table 4.3.\nExample. There are three word groups with the same name: cm-0, cm-1, cm-2 Because they all refer to separate entities, they all have a unique number. 25The numbers correspond to the word maps sentence and token index (sentence, index)."
      }, {
        "heading" : "4. Triples 45",
        "text" : "Example. In Table 4.2 the predicate connect occurs several times as (potentially distinct) word maps. In Table 4.3 we reduced them all to a single :connect.\nTo allow clients to access the reified triple, we update the knowledge base from Section 3.4 with a new relation:\n(triple t)\ntriple is a unary relation and we simply add every triple we found to it. Clients can then access any information from the triple by unifying with t. A reified triple is a map that has the following schema:\n{:subject {:symbol word group name :group vector of word maps}\n:predicate predicate :object same layout as :subject}"
      }, {
        "heading" : "46 4.2. Reifying triples",
        "text" : ""
      }, {
        "heading" : "4. Triples 47",
        "text" : ""
      }, {
        "heading" : "48 4.2. Reifying triples",
        "text" : ""
      }, {
        "heading" : "4. Triples 49",
        "text" : ""
      }, {
        "heading" : "4.3. Exporting an OWL ontology",
        "text" : "We create an OWL class for every subject and object word group of a triple. OWL 2 introduces a feature called punning [9] where we create an individual for each of our classes and then use object properties on these individuals to describe relationships between the classes.\nBecause a triple’s predicate describes a relationship between its subject and object, we use an object property with a name based on the predicate and add an axiom to the ontology that links the subject’s individual with the object’s individual via this object property.\nA subject or object word group contains word maps that have additional information about that group, such as all of the actual words (tokens) that make up the group. We add this information as datatype properties to the subject’s or object’s individual.\nFigure 4.1 shows a subset of an ontology that is based on Table 4.3. The “has individual” loops are an artifact of our use of punning. Even though node-start-0, node-noname-0 and num-10-0 are only shown as individuals, they are still represented as classes (and as subclasses of Thing) in the full ontology.\nExample. We can query the ontology using SparQL. E.g. to return all subject word groups with their constituent tokens that are linked to the word group distance-0, we can run the following query:\n1 PREFIX : <http://clide.informatik.uni-bremen.de/clide-nlp#> 2 SELECT ?subject ?token WHERE { 3 ?subject :have :distance-0. 4 ?subject :hasToken ?token 5 }\nQuery result: ?subject ?token node-noname-0 “Noname” node-noname-0 “Node” node-noname-0 “node” node-start-0 “Start” node-start-0 “node”\n51\n5. Use case: Graph creation from a natural language\nspecification\nThis chapter introduces an example application that can create graphs from a text. The application is directly integrated into clide-nlp and uses the reified triples introduced in Section 4.2 as its input.\nThe application and its input can be seen in the reconciler’s dependency graph Figure 3.2 and is named :draw there.\nThe output is a graph with the nodes and edges as described in the text and a list of warnings about ambiguous or incomplete information extracted from the triples.\nThe application should detect\n• named nodes,\n• edges that are directly specified with a name,\n• edges that are indirectly specified as a connection between two nodes, and\n• edge lengths.\nEmit warnings when there are\n• simple contradictions, like different lengths for the same edge,\n• unfinished edge or node specifications,\n• sentences that specify the same node or edge twice, and\n• non-integer edge lengths.\nTo achieve these goals, we must specify what kind of sentences we would like to understand. We can specify edges with the following sentences:\nType Sentence example Unnamed edges Node A and Node B are connected. Named edges Edge B starts at Node A and goes to Node B. Edge distance Edge B is 5 cm long. Edge distance Node A and Node B have a distance of 5 cm.\nNodes are specified implicitly by mentioned e.g. “Node A” somewhere in the text.\nDistances have a unit and magnitude. We only support “cm” as a unit.\nBecause we use triples as the basis of our application, we can structure the sentences differently,"
      }, {
        "heading" : "52 5.1. Triple walks",
        "text" : "while keeping the triple set the same.\nExample. Extracting triples from the sentences\nNode A and Node B are connected. Node A and Node B have a distance of 5 cm.\nand the sentence\nNode A and Node B are connected with a distance of 5 cm.\nwill result in the same set of triples."
      }, {
        "heading" : "5.1. Triple walks",
        "text" : "We define some helper goals that let us define a walk that follows a chain of triples, and allows us to essentially pattern match on that chain.\n(subjecto→ t & clauses)\nA subject walk succeeds if t = (S0, P0, O0) can satisfy each of the clauses. t is the triple we start with.\nA clause can be a tuple with a predicate and object or a 3-tuple in which case the last element is a partial match of a word map in the S0’s word group. We thread the subject of t through each of the clauses. This is illustrated in Figure 5.1."
      }, {
        "heading" : "5. Use case: Graph creation from a natural language specification 53",
        "text" : "Example. We define the following subject walk to find an edge: S0 :start O1 where a word map in S0 must match {:lemma \"Edge\"} S0 :at O1 S0 :go O2 S0 :to O2\nWe use O1 and O2 in multiple clauses to make sure that the clauses only match the same object group. Looking at Table 4.3 we can find a triple for which the walk succeeds: t = (edge-test-0, :start, node-start-0)\nedge-test-0 :start node-start-0 edge-test-0 :at node-start-0 edge-test-0 :go end-node-0 edge-test-0 :to end-node-0\nWe can run that walk with subjecto→ inside a core.logic query:\n1 (subjecto→ t 2 [:start O1 {:lemma \"Edge\"}] 3 [:at O1] 4 [:go O2] 5 [:to O2])\nIf the subjecto→ goal succeeds, O1 will be bound to node-start-node-0 and O2 to end-node-0. We can then extract more information out of them, if necessary and perform some additional validation.\n(objecto→ t & clauses)\nThe counterpart to subjecto→ that matches on the triples’ objects first (see Figure 5.2)."
      }, {
        "heading" : "54 5.1. Triple walks",
        "text" : "Example. We define an object walk to get the distance between the nodes of our edge. We assume that the previous subject walk succeeded with t = (S0, P0, O0).\nO0 :have O3 where a word map in O3 must match {:lemma \"distance\"} O3 :of O4 O4 :be O5 where a word map in O5 must match {:tag \"CD\"}\nThe walk succeeds, if we start with the triple t = (edge-test-0, :start, node-start-0) and follow the triples:\nnode-start-0 :have distance-0 distance-0 :of cm-2\ncm-2 :be num-10-0\nWe can run that walk with objecto→ inside a core.logic query:\n1 (objecto→ t 2 [:have O3 {:lemma \"distance\"}] 3 [:of O4] 4 [:be O5 {:tag \"CD\"}])\nIf the objecto→ goal succeeds and O3, O4, and O5 are logic variables, they will be bound to distance-0, cm-2, and num-10-0 respectively. We can then extract more information out of them, if necessary."
      }, {
        "heading" : "5. Use case: Graph creation from a natural language specification 55",
        "text" : ""
      }, {
        "heading" : "5.2. Implementation",
        "text" : "We implement several collection stages that search the triples for relevant information. The general process follows these steps:\n1. Collect node\n2. Collect edges\n3. Check found edges for inconsistencies\n4. Check for singleton nodes\nEach stage might emit warnings about inconsistencies, which we need to present to the user later.\nThe node and edge collecting stages make use of subject and object walks to extract the information that we need to create a graph.\nExample. If we combine the object walk with the subject walk in our previous examples, we get an edge with the distance between its nodes:\n• The group S0 = edge-test-0 contains the edge label (Test)\n• The group O5 = num-10-0 is the edge’s length\n• The group O4 = cm-2 contains the edge length’s unit\nAn edge’s or node’s label is extracted by using the word that is immediately next to it in the original text. Extracting the edge’s length and unit is easy, because O5 and O4 are word groups with only a single word map, so we can simply use that word map’s lemma. We then check if the length is an integer and the unit is one of the supported units (cm).\nThe output of our example application is presented to the users as additional annotations:\ndraw Level: Chunk\nShows the graph that was extracted from the chunk’s text. The input text contains some ambiguities, which means the graph is not looking like we want it to look. The double edge between nodes “Start” and “End” looks especially suspicious."
      }, {
        "heading" : "56 5.2. Implementation",
        "text" : "draw-warnings and draw-warning-highlights Level: Chunk\nThe information collecting stages emit warnings about potential problem areas in our input text. We display some information about which sentences and words might be problematic and some suggestions about how to resolve the warnings. Here we can see that the edge “Test” is specified twice which explains the double edge we saw earlier.\n57\n6. Conclusion\nWe have shown how we can extract information from a text in a straightforward way, while only using the simple tools provided by CoreNLP, and what kind of problems result in integrating such a system into a distributed development environment like Clide.\nTo achieve our initial goals, we have taken the following approach:\n1. We provide an underlying NLP knowledge base (see Section 3.3) for an input text based on CoreNLP’s dependency parser and its coreference resolution system.\n2. The reconciler (see Section 3.2) makes sure that text changes sent by Clide are integrated into the knowledge base, and keeps the ontology and annotations up to date and in sync with the input text while making sure to only update them when really necessary. Because coreference resolution is a slow process, and we need to rerun it after each text change, we split the input text into chunks and only operate on one chunk at a time.\n3. Triple builders (see Chapter 4) are simple queries on the knowledge base that extract meaningful units of information from the text’s semantic graphs in the form of triples (subject, predicate, object).\n4. The triples extracted from the queries are augmented by grouping their subjects and objects according to the coreference chain they belong to, yielding an ontology with unique classes (word groups). We provide an example for how to export an OWL ontology (see Section 4.3), so that the ontology can be used by other tools.\n5. We expose the underlying structure of sentences and texts to the user by providing annotations that visualize that structure in Clide (see Section 3.5).\n6. In Chapter 5 we have shown how the ontology can be used to create graphs from a simple natural language specification and how using triples enables sentences in the input text with slightly different phrasing to still yield the same ontology and graph.\nThere are some (solvable) caveats to our approach:\n• Some triples we extract make no sense. This is a symptom of missing information and of the triple builders’ simplistic nature. E.g. some triple builders like nsubjpass-ccomp are limited, because there is currently no way to determine a verb’s transitivity.\n• At the moment we ignore all tenses and merge potentially different states of word groups in different time frames (as indicated e.g. by a triple’s predicate’s tense) into one ontology. Using a verb ontology in combination with the predicates’ part-of-speech tags, we could split the ontology into separate ontologies, one for each time frame. We could then e.g.\n58\ntrack the changes of a word group’s attributes over time.\n• Currently the ontology we create has no concept hierarchy. We could integrate WordNet to group word groups that have the same concept behind them under one umbrella via e.g. is-a relationships in the ontology.\n• At the moment the ontologies we extract are limited to one chunk of a text only. It should be possible to develop some heuristics that would allow us to merge the ontologies of two or more chunks together into one ontology.\n• clide-nlp is limited by Clide’s current behavior to keep annotations static and to limit the assistants direct influence to the server-side only. A client side integration could enable some interactive aspects, like e.g. defining a new triple builder on the fly and using Clojure’s dynamic aspects to make it available to the system immediately.26\nThe tools we created while building the assistant are general enough to be used outside of Clide and could be integrated into other editing environments. Our NLP knowledge base with information from CoreNLP’s semantic graphs, its coreference resolution system, and the ontology we extracted from them, provides easy access to information about a text.\nExposing Clojure’s and clide-nlp’s dynamic natures in Clide’s interface and combining it with Clide’s collaborative aspects can enable an environment and framework where we can quickly and collaboratively develop simple systems that have some albeit limited and domain specific text understanding.\n26This is possible already, but not exposed to the user in the UI.\n59\nA. Part of Speech Tags\nThe part of speech tags used by CoreNLP are based on the tags used by the Penn Treebank [15]. There are however some differences. Table A.1 shows an updated version of the part of speech tag table in [15, p. 317] based on experience with CoreNLP. As such this table is most likely incomplete, but enough to follow the examples in this report.\n61\nB. Installation notes\nThe CD that accompanies this report contains three ZIP files:\nFilename Contents report.pdf A digital copy of this report clide-nlp-src.zip The source code for clide-nlp clide-src.zip The source code to a version of Clide that works correctly with clide-nlp clide-nlp.zip Contains an executable JAR of clide-nlp and startup scripts\nclide-nlp requires Java 7 and Clide works best with a WebKit-based browser like e.g. Chrome. You need approx. 3 GiB of RAM to successfully run clide-nlp. To execute it:\n• Extract clide-nlp.zip and run run.sh on Linux/FreeBSD or run.bat on Windows.\n• Wait a minute or two.\n• A launcher window will pop up that informs you about the startup process.\n• After the system is ready, clide-nlp should inform you that it is ready at http://localhost: 14000\n• Open the URL and log in with user clide-nlp and password clide-nlp.\n• Open the clide-nlp/Example project and look at 00-README.txt for further help.\n• And most importantly, try editing one of the files!\n63\nList of Figures\n1.1 The clide-nlp annotation semantic-graph showing the semantic graph for the current sentence (highlighted in blue). . . . . . . . . . . . . . . . . . . . . . . . . . . 9 1.2 The clide-nlp annotation reified-triples showing the triples that are found for the given text and the annotation same-as highlighting all words that are detected as belonging together in red. The highlighted words are part of the group Edge-Test-it-0. 10 1.3 Showing all annotation from the example application draw, draw-warnings and draw-warning-highlights that draws the graph specified in the text. . . . . . . . . . 11\n3.1 Architecture overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 3.2 Chunk annotation dependency graph . . . . . . . . . . . . . . . . . . . . . . . . . 22\n4.1 A view on the ontology that is extracted from Table 4.3 . . . . . . . . . . . . . . 49\n5.1 Illustration of how a subject walk beginning with triple t, threads t’s subject through the whole triple chain. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 5.2 Illustration of how an object walk beginning with triple t, threads the object of each consecutive triple through the triple chain. . . . . . . . . . . . . . . . . . . . 53\n65"
      }, {
        "heading" : "66 Bibliography",
        "text" : "[13] Lord, P. The Semantic Web takes Wing: Programming Ontologies with Tawny-OWL. arXiv preprint arXiv:1303.0213 (2013).\n[14] Lüth, C., and Ring, M. A web interface for Isabelle: The Next Generation. In Intelligent Computer Mathematics. Springer, 2013, pp. 326–329.\n[15] Marcus, M. P., Marcinkiewicz, M. A., and Santorini, B. Building a large annotated corpus of English: The Penn Treebank. Computational linguistics 19, 2 (1993), 313–330.\n[16] Miller, G. A. WordNet: A Lexical Database for English. Communications of the ACM 38, 11 (1995), 39–41.\n[17] Raghunathan, K., Lee, H., Rangarajan, S., Chambers, N., Surdeanu, M., Jurafsky, D., and Manning, C. A multi-pass sieve for coreference resolution. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing (2010), Association for Computational Linguistics, pp. 492–501.\n[18] Recasens, M., de Marneffe, M.-C., and Potts, C. The life and death of discourse entities: Identifying singleton mentions. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2013), pp. 627–633.\n[19] Ring, M. Eine webbasierte Entwicklungsumgebung für den interaktiven Theorembeweiser Isabelle. Diplomarbeit, Universität Bremen, 2013.\n[20] Ring, M., and Lüth, C. Collaborative Interactive Theorem Proving with Clide. In Proceedings of ITP 2014 (2014), Springer.\n[21] Varjú, Z., Littauer, R., and Ernis, P. Using Clojure in Linguistic Computing. In Proceedings of the 5th European Lisp Symposium (2012).\n[22] Wynne, M., and Hellesoy, A. The Cucumber Book: Behaviour-driven Development for Testers and Developers. Pragmatic Bookshelf, 2012."
      } ],
      "references" : [ {
        "title" : "DBpedia: A Nucleus for a Web of Open Data",
        "author" : [ "S. Auer", "C. Bizer", "G. Kobilarov", "J. Lehmann", "R. Cyganiak", "Z. Ives" ],
        "venue" : "In The Semantic Web. Springer,",
        "citeRegEx" : "1",
        "shortCiteRegEx" : "1",
        "year" : 2007
      }, {
        "title" : "Relational programming in miniKanren: Techniques, applications, and implementations",
        "author" : [ "W.E. Byrd" ],
        "venue" : "PhD thesis, Indiana University,",
        "citeRegEx" : "2",
        "shortCiteRegEx" : "2",
        "year" : 2010
      }, {
        "title" : "VerbOcean: Mining the Web for Fine-Grained Semantic Verb Relations",
        "author" : [ "T. Chklovski", "P. Pantel" ],
        "venue" : "In EMNLP (2004),",
        "citeRegEx" : "3",
        "shortCiteRegEx" : "3",
        "year" : 2004
      }, {
        "title" : "Generating typed dependency parses from phrase structure parses",
        "author" : [ "De Marneffe", "M.-C", "B. MacCartney", "Manning", "C. D" ],
        "venue" : "In Proceedings of LREC (2006),",
        "citeRegEx" : "4",
        "shortCiteRegEx" : "4",
        "year" : 2006
      }, {
        "title" : "Stanford typed dependencies manual",
        "author" : [ "de Marneffe", "M.-C", "C.D. Manning" ],
        "venue" : "Revised in December 2013 for the Stanford Parser v",
        "citeRegEx" : "5",
        "shortCiteRegEx" : "5",
        "year" : 2008
      }, {
        "title" : "The Joy of Clojure: Thinking the Clojure Way",
        "author" : [ "M. Fogus", "C. Houser" ],
        "venue" : "Manning Publications Co.,",
        "citeRegEx" : "7",
        "shortCiteRegEx" : "7",
        "year" : 2011
      }, {
        "title" : "The Reasoned Schemer",
        "author" : [ "D.P. Friedman", "W.E. Byrd", "O. Kiselyov" ],
        "venue" : null,
        "citeRegEx" : "8",
        "shortCiteRegEx" : "8",
        "year" : 2005
      }, {
        "title" : "OWL 2 Web Ontology Language: New Features and Rationale",
        "author" : [ "C. Golbreich", "E.K. Wallace", "P.F. Patel-Schneider" ],
        "venue" : null,
        "citeRegEx" : "9",
        "shortCiteRegEx" : "9",
        "year" : 2012
      }, {
        "title" : "Semantic Web: Grundlagen",
        "author" : [ "P. Hitzler" ],
        "venue" : "eXamen.press. Springer-Verlag, Berlin,",
        "citeRegEx" : "10",
        "shortCiteRegEx" : "10",
        "year" : 2008
      }, {
        "title" : "Stanford’s multi-pass sieve coreference resolution system at the CoNLL-2011 shared task",
        "author" : [ "H. Lee", "Y. Peirsman", "A. Chang", "N. Chambers", "M. Surdeanu", "D. Jurafsky" ],
        "venue" : "In Proceedings of the Fifteenth Conference on Computational Natural Language Learning: Shared Task",
        "citeRegEx" : "12",
        "shortCiteRegEx" : "12",
        "year" : 2011
      }, {
        "title" : "The Semantic Web takes Wing: Programming Ontologies with Tawny-OWL",
        "author" : [ "P. Lord" ],
        "venue" : "arXiv preprint arXiv:1303.0213",
        "citeRegEx" : "13",
        "shortCiteRegEx" : "13",
        "year" : 2013
      }, {
        "title" : "A web interface for Isabelle: The Next Generation",
        "author" : [ "C. Lüth", "M. Ring" ],
        "venue" : "In Intelligent Computer Mathematics. Springer,",
        "citeRegEx" : "14",
        "shortCiteRegEx" : "14",
        "year" : 2013
      }, {
        "title" : "Building a large annotated corpus of English: The Penn Treebank",
        "author" : [ "M.P. Marcus", "M.A. Marcinkiewicz", "B. Santorini" ],
        "venue" : "Computational linguistics 19,",
        "citeRegEx" : "15",
        "shortCiteRegEx" : "15",
        "year" : 1993
      }, {
        "title" : "WordNet: A Lexical Database for English",
        "author" : [ "G.A. Miller" ],
        "venue" : "Communications of the ACM 38,",
        "citeRegEx" : "16",
        "shortCiteRegEx" : "16",
        "year" : 1995
      }, {
        "title" : "A multi-pass sieve for coreference resolution",
        "author" : [ "K. Raghunathan", "H. Lee", "S. Rangarajan", "N. Chambers", "M. Surdeanu", "D. Jurafsky", "C. Manning" ],
        "venue" : "In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing",
        "citeRegEx" : "17",
        "shortCiteRegEx" : "17",
        "year" : 2010
      }, {
        "title" : "The life and death of discourse entities: Identifying singleton mentions",
        "author" : [ "M. Recasens", "de Marneffe", "M.-C", "C. Potts" ],
        "venue" : "In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies",
        "citeRegEx" : "18",
        "shortCiteRegEx" : "18",
        "year" : 2013
      }, {
        "title" : "Eine webbasierte Entwicklungsumgebung für den interaktiven Theorembeweiser Isabelle",
        "author" : [ "M. Ring" ],
        "venue" : "Diplomarbeit, Universität Bremen,",
        "citeRegEx" : "19",
        "shortCiteRegEx" : "19",
        "year" : 2013
      }, {
        "title" : "Collaborative Interactive Theorem Proving with Clide",
        "author" : [ "M. Ring", "C. Lüth" ],
        "venue" : "In Proceedings of ITP",
        "citeRegEx" : "20",
        "shortCiteRegEx" : "20",
        "year" : 2014
      }, {
        "title" : "Using Clojure in Linguistic Computing",
        "author" : [ "Z. Varjú", "R. Littauer", "P. Ernis" ],
        "venue" : "In Proceedings of the 5th European Lisp Symposium",
        "citeRegEx" : "21",
        "shortCiteRegEx" : "21",
        "year" : 2012
      }, {
        "title" : "The Cucumber Book: Behaviour-driven Development for Testers and Developers",
        "author" : [ "M. Wynne", "A. Hellesoy" ],
        "venue" : "Pragmatic Bookshelf,",
        "citeRegEx" : "22",
        "shortCiteRegEx" : "22",
        "year" : 2012
      } ],
      "referenceMentions" : [ {
        "referenceID" : 19,
        "context" : "Scenarios consist of steps and each step is parsed using a user provided regular expression [22].",
        "startOffset" : 92,
        "endOffset" : 96
      }, {
        "referenceID" : 11,
        "context" : "It was originally intended to be a web-based development environment for Isabelle4 only [14, 19].",
        "startOffset" : 88,
        "endOffset" : 96
      }, {
        "referenceID" : 16,
        "context" : "It was originally intended to be a web-based development environment for Isabelle4 only [14, 19].",
        "startOffset" : 88,
        "endOffset" : 96
      }, {
        "referenceID" : 16,
        "context" : "In contrast with previous Isabelle interfaces, it provides better visualization of the prover’s results than traditional and sequential REPL5-based interfaces through leveraging Web technologies like HTML5 and JavaScript [19].",
        "startOffset" : 221,
        "endOffset" : 225
      }, {
        "referenceID" : 17,
        "context" : "It has since undergone further development and has evolved to facilitate collaborative editing of documents with support for other languages besides Isabelle [20].",
        "startOffset" : 158,
        "endOffset" : 162
      }, {
        "referenceID" : 17,
        "context" : "It uses an approach called Universal Collaboration where an assistant is seen as just an additional collaborator by the system [20].",
        "startOffset" : 127,
        "endOffset" : 131
      }, {
        "referenceID" : 17,
        "context" : "While Clide is distributive and asynchronous in nature, it provides an interface that can be used to implement assistants “in a simple, synchronous manner” [20].",
        "startOffset" : 156,
        "endOffset" : 160
      }, {
        "referenceID" : 5,
        "context" : "Clojure provides a REPL which allows us to patch in new code into a running system, enabling a small “thought-code-feedback loop” [7].",
        "startOffset" : 130,
        "endOffset" : 133
      }, {
        "referenceID" : 0,
        "context" : "It provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).",
        "startOffset" : 163,
        "endOffset" : 170
      }, {
        "referenceID" : 1,
        "context" : "It provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).",
        "startOffset" : 163,
        "endOffset" : 170
      }, {
        "referenceID" : 2,
        "context" : "It provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).",
        "startOffset" : 163,
        "endOffset" : 170
      }, {
        "referenceID" : 0,
        "context" : "It provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).",
        "startOffset" : 174,
        "endOffset" : 181
      }, {
        "referenceID" : 1,
        "context" : "It provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).",
        "startOffset" : 174,
        "endOffset" : 181
      }, {
        "referenceID" : 2,
        "context" : "It provides built-in persistent data structures with their own reader syntax, that are immutable and have clear and intuitive value equality semantics (1 is 1 and [1 2 3] is [1 2 3]).",
        "startOffset" : 174,
        "endOffset" : 181
      }, {
        "referenceID" : 18,
        "context" : "This allows us to leverage all existing Java NLP libraries [6, 21].",
        "startOffset" : 59,
        "endOffset" : 66
      }, {
        "referenceID" : 5,
        "context" : "More information on Clojure is available in [6, 7].",
        "startOffset" : 44,
        "endOffset" : 50
      }, {
        "referenceID" : 1,
        "context" : "Byrd as part of his PhD thesis [2].",
        "startOffset" : 31,
        "endOffset" : 34
      }, {
        "referenceID" : 18,
        "context" : "logic when we need it and use Clojure functional programming aspects otherwise [21].",
        "startOffset" : 79,
        "endOffset" : 83
      }, {
        "referenceID" : 0,
        "context" : "List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query (project [q] .",
        "startOffset" : 27,
        "endOffset" : 34
      }, {
        "referenceID" : 1,
        "context" : "List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query (project [q] .",
        "startOffset" : 27,
        "endOffset" : 34
      }, {
        "referenceID" : 2,
        "context" : "List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query (project [q] .",
        "startOffset" : 27,
        "endOffset" : 34
      }, {
        "referenceID" : 0,
        "context" : "List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query (project [q] .",
        "startOffset" : 102,
        "endOffset" : 109
      }, {
        "referenceID" : 1,
        "context" : "List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query (project [q] .",
        "startOffset" : 102,
        "endOffset" : 109
      }, {
        "referenceID" : 2,
        "context" : "List membership (member◦ q [1 2 3]) A goal that succeeds if q is bound to value that is in the vector [1 2 3] Extract a logic variable’s value inside a query (project [q] .",
        "startOffset" : 102,
        "endOffset" : 109
      }, {
        "referenceID" : 0,
        "context" : "Domain constraint (in q (interval 1 10)) Makes sure that q is bound to a value in the interval [1, 10].",
        "startOffset" : 95,
        "endOffset" : 102
      }, {
        "referenceID" : 8,
        "context" : "Domain constraint (in q (interval 1 10)) Makes sure that q is bound to a value in the interval [1, 10].",
        "startOffset" : 95,
        "endOffset" : 102
      }, {
        "referenceID" : 1,
        "context" : "logic code shown here is based on the code presentation in [2, 8].",
        "startOffset" : 59,
        "endOffset" : 65
      }, {
        "referenceID" : 6,
        "context" : "logic code shown here is based on the code presentation in [2, 8].",
        "startOffset" : 59,
        "endOffset" : 65
      }, {
        "referenceID" : 1,
        "context" : "Presentation Actual code <x>◦ <x>o A goal is written with a suffix o to distinguish it from already defined functions on the functional programming side, while making clear that they have the same outcome in both paradigms [2].",
        "startOffset" : 223,
        "endOffset" : 226
      }, {
        "referenceID" : 6,
        "context" : "conda conda ≡ == 6≡ != The Reasoned Schemer [8] provides a good introduction to miniKanren and in extension also core.",
        "startOffset" : 44,
        "endOffset" : 47
      }, {
        "referenceID" : 10,
        "context" : "Tawny-OWL Tawny-OWL11 is a Clojure library that provides a domain specific language for building OWL ontologies [13].",
        "startOffset" : 112,
        "endOffset" : 116
      }, {
        "referenceID" : 8,
        "context" : "querying them via SparQL [10].",
        "startOffset" : 25,
        "endOffset" : 29
      }, {
        "referenceID" : 3,
        "context" : "The dependency parser makes the underlying structures of sentences visible in the form of grammatical relations between sentence parts [4].",
        "startOffset" : 135,
        "endOffset" : 138
      }, {
        "referenceID" : 4,
        "context" : "The grammatical relations are described in [5].",
        "startOffset" : 43,
        "endOffset" : 46
      }, {
        "referenceID" : 3,
        "context" : "The dependency parser can collapse prepositions and coordinations into grammatical relations [4, 5].",
        "startOffset" : 93,
        "endOffset" : 99
      }, {
        "referenceID" : 4,
        "context" : "The dependency parser can collapse prepositions and coordinations into grammatical relations [4, 5].",
        "startOffset" : 93,
        "endOffset" : 99
      }, {
        "referenceID" : 12,
        "context" : "Its tagset is based on the POS tagset used by the Penn Treebank, described in [15].",
        "startOffset" : 78,
        "endOffset" : 82
      }, {
        "referenceID" : 9,
        "context" : "It was introduced in [11, 12, 17, 18].",
        "startOffset" : 21,
        "endOffset" : 37
      }, {
        "referenceID" : 14,
        "context" : "It was introduced in [11, 12, 17, 18].",
        "startOffset" : 21,
        "endOffset" : 37
      }, {
        "referenceID" : 15,
        "context" : "It was introduced in [11, 12, 17, 18].",
        "startOffset" : 21,
        "endOffset" : 37
      }, {
        "referenceID" : 9,
        "context" : "It competed in the CoNLL Shared Task 2011, where it achieved the highest score in both the closed and open tracks [12].",
        "startOffset" : 114,
        "endOffset" : 118
      }, {
        "referenceID" : 17,
        "context" : "There are three operations [20]: • Retain(n) • Insert(s) • Delete(n) Since Clide is written in Scala Retain, Insert and Delete are implemented using Scala case classes.",
        "startOffset" : 27,
        "endOffset" : 31
      }, {
        "referenceID" : 4,
        "context" : "relation can be either a string containing a typed dependency relation (see [5]) or if the relation is a collapsed relation, a vector of the first and second part of the relation (e.",
        "startOffset" : 76,
        "endOffset" : 79
      }, {
        "referenceID" : 4,
        "context" : "nn is the noun compound modifier dependency relation that asserts that one noun modifies another noun [5].",
        "startOffset" : 102,
        "endOffset" : 105
      }, {
        "referenceID" : 17,
        "context" : "Ignoring annotations, we can treat :plain and :annotate as :retain operations [20].",
        "startOffset" : 78,
        "endOffset" : 82
      }, {
        "referenceID" : 13,
        "context" : "While the triple builders only extracts information from semantic graphs, they could be extended to include information from ontologies like WordNet [16], VerbOcean [3] or DBpedia [1] to further constrain the triples that are found.",
        "startOffset" : 149,
        "endOffset" : 153
      }, {
        "referenceID" : 2,
        "context" : "While the triple builders only extracts information from semantic graphs, they could be extended to include information from ontologies like WordNet [16], VerbOcean [3] or DBpedia [1] to further constrain the triples that are found.",
        "startOffset" : 165,
        "endOffset" : 168
      }, {
        "referenceID" : 0,
        "context" : "While the triple builders only extracts information from semantic graphs, they could be extended to include information from ontologies like WordNet [16], VerbOcean [3] or DBpedia [1] to further constrain the triples that are found.",
        "startOffset" : 180,
        "endOffset" : 183
      }, {
        "referenceID" : 4,
        "context" : "The meaning of a subject can be modified with an adjectival modifier (amod) [5].",
        "startOffset" : 76,
        "endOffset" : 79
      }, {
        "referenceID" : 4,
        "context" : "Verbs and adjectives have an acomp (adjectival complement) relation, if an adjective can be treated as the verb’s object [5].",
        "startOffset" : 121,
        "endOffset" : 124
      }, {
        "referenceID" : 4,
        "context" : "enter does not have its own subject but refers to the subject of managed (He) [5].",
        "startOffset" : 78,
        "endOffset" : 81
      }, {
        "referenceID" : 4,
        "context" : "An agent is “introduced by the preposition by” [5].",
        "startOffset" : 47,
        "endOffset" : 50
      }, {
        "referenceID" : 4,
        "context" : "This triple builder captures numeric modifiers (num) [5] of nouns, e.",
        "startOffset" : 53,
        "endOffset" : 56
      }, {
        "referenceID" : 7,
        "context" : "OWL 2 introduces a feature called punning [9] where we create an individual for each of our classes and then use object properties on these individuals to describe relationships between the classes.",
        "startOffset" : 42,
        "endOffset" : 45
      }, {
        "referenceID" : 12,
        "context" : "The part of speech tags used by CoreNLP are based on the tags used by the Penn Treebank [15].",
        "startOffset" : 88,
        "endOffset" : 92
      } ],
      "year" : 2014,
      "abstractText" : null,
      "creator" : "LaTeX with hyperref package"
    }
  }