Venue: ACL 2017

# Generic Axiomatization of Families of Noncrossing Graphs in

Dependency Parsing

Anssi Yli-Jyra

University of Helsinki, Finland

anssi.yli-jyra@helsinki.fi &Carlos Gomez-Rodriguez

Universidade da Coruna, Spain

carlos.gomez@udc.es

###### Abstract

We present a simple encoding for unlabeled noncrossing graphs and show how its latent counterpart helps us to represent several families of directed and undirected graphs used in syntactic and semantic parsing of natural language as context-free languages. The families are separated purely on the basis of forbidden patterns in latent encoding, eliminating the need to differentiate the families of non-crossing graphs in inference algorithms: one algorithm works for all when the search space can be controlled in parser input.

## 1 Introduction

Dependency parsing has received wide attention in recent years, as accurate and efficient dependency parsers have appeared that are applicable to many languages. Traditionally, dependency parsers have produced syntactic analyses in tree form, including exact inference algorithms that search for maximum projective trees Eisner and Satta (1999) and maximum spanning trees McDonald et al. (2005) in weighted digraphs, as well as greedy and beam-search approaches that forgo exact search for extra efficiency Zhang and Nivre (2011).

Recently, there has been growing interest in providing a richer analysis of natural language by going beyond trees. In semantic dependency parsing Oepen et al. (2015); Kuhlmann and Oepen (2016), the desired syntactic representations can have indegree greater than \(1\) (re-entrancy), suggesting the search for maximum acyclic subgraphs Schluter (2014, 2015). As this inference task is intractable Guruswami et al. (2011), noncrossing digraphs have been studied instead, e.g. by Kuhlmann and Johnsson (2015) who provide a \(O(n^{3})\) parser for maximum noncrossing acyclic subgraphs.

Yli-Jyra (2005) studied how to axiomatize dependency trees as a special case of noncrossing digraphs. This gave rise to a new homomorphic representation of context-free languages that proves the classical Chomsky and Schutzenberger theorem using a quite different internal language. In this language, the brackets indicate arcs in a dependency tree in a way that is reminiscent to encoding schemes used earlier by Greibach (1973) and Olazer (2003). Cubic-time parsing algorithms that are incidentally or intentionally applicable to this kind of homomorphic representations have been considered, e.g., by Nederhof and Satta (2003), Hulden (2011), and Yli-Jyra (2012).

Extending these insights to arbitrary noncrossing digraphs, or to relevant families of them, is far from obvious. In this paper, we develop (1) a _linear encoding_ supporting general noncrossing digraphs, and (2) show that the encoded noncrossing digraphs form a _context-free language_. We then give it (3) two _homomorphic, nondervative representations_ and use the latent local features of the latter to characterize various families of digraphs.

Apart from the obvious relevance to the theory of context-free languages, this contribution has the practical potential to enable (4) _generic context-free parsers_ that produce different families of noncrossing graphs with the same set of inference rules while the search space in each case is restricted with lexical features and the grammar.

OutlineAfter some background on graphs and parsing as inference (Section 2), we use an **ontology of digraphs** to illustrate natural families of noncrossing digraphs in Section 3. We then develop, in Section 4, the first **latent context-free representation** for the set of noncrossing digraphs, then extended in Section 5 with additional latent states supporting our **finite-state axiomatization** of digraph properties, and allowing us to

control the search space using the lexicon. The **experiments** in Section 6 cross-validate our axioms and sample the growth of the constrained search spaces. Section 7 outlines the applications for practical parsing, and Section 8 concludes.

## 2 Background

Graphs and DigraphsA _graph_ is a pair \((V,E)\) where \(V\) is a finite set of vertices and \(E\subseteq\{\{u,v\}\subseteq V\}\) is a set of edges. A sequence of edges of the form \(\{v_{0},v_{1}\},~{}\{v_{1},v_{2}\},~{}...,\\ \{v_{m-1},v_{m}\}\), with no repetitions in \(v_{1},...,v_{m}\), is a _path_ between vertices \(v_{0}\) and \(v_{m}\) and _empty_ if \(m=0\). A graph is a _forest_ if no vertex has a non-empty path to itself and _connected_ if all pairs of vertices have a path. A _tree_ is a connected forest.

A _digraph_ is a pair \((V,A)\) where \(A\subseteq V\times V\) is a set of arcs \(u\to v\), thus a _directed_ graph. Its _underlying graph_, \((V,E_{A})\), has edges \(E_{A}=\{\{u,v\}\mid(u,v)\in A\}\). A sequence of arcs \(v_{0}\to v_{1},v_{1}\to v_{2},...,v_{m-1}\to v_{m}\), with no repetitions in \(v_{1},\ldots,v_{m}\), is a _directed path_, and _empty_ if \(m=0\).

A digraph without self-loops \(v\to v\) is _loop-free_ (property \(\mathbf{DIG RAPH_{LF}}\)). We will focus on loop-free digraphs unless otherwise specified, and denote them just by \(\mathbf{DIGRAPH}\) for brevity. A digraph is d-acyclic (\(\mathbf{ACYC_{D}}\)), Aka a _dag_ if no vertex has a non-empty directed path to itself, _u-acyclic_ (\(\mathbf{ACYC_{U}}\)) Aka a _m(ixed)-forest_ if its underlying graph is a forest, and _weakly connected_ (w.c., \(\mathbf{CONN_{W}}\)) if its underlying graph is connected.

Dependency ParsingThe _complete digraph_\(G_{S}(V,A)\)_of a sentence_\(S=x_{1}...x_{n}\) consists of vertices \(V=\{1,...,n\}\) and all possible arcs \(A=V\times V-\{(i,i)\}\). The vertex \(i\in V\) corresponds to the word \(x_{i}\) and the arc \(i\to j\in A\) corresponds to a possible dependency between the words \(x_{i}\) and \(x_{j}\).

The task of _dependency parsing_ is to find a constrained subgraph \(G^{\prime}_{S}(V,A^{\prime})\) of the complete digraph \(G_{S}\) of the sentence. The standard solution is a rooted directed tree called a _dependency tree_ or a dag called a _dependency graph_.

Constrained InferenceIn _arc-factored parsing_McDonald et al. (2005), each possible arc \(i\to j\) is equipped with a positive weight \(w_{ij}\), usually computed as a weighted sum \(w_{ij}=\mathbf{w}\cdot\Phi(S,i\to j)\) where \(\mathbf{w}\) is a weight vector and \(\Phi(\mathbf{x},i\to j)\) a feature vector extracted from the sentence \(\mathbf{x}\), considering the dependency relation from word \(x_{i}\) to word \(x_{j}\). Parsing then consists in finding an arc subset \(A^{\prime}\subseteq A\) that gives us a constrained subgraph \((V,A^{\prime})\in\text{Constrained}(V,A)\) of the complete digraph \((V,A)\) with maximum sum of arc weights:

\[(V,A^{\prime})=\operatorname*{arg\,max}_{(VA^{\prime})\;\in\;\text{Constrained }(V,A)}~{}\sum_{i\to j\in A^{\prime}}w_{i,j}.\]

The complexity of this inference task depends on the constraints imposed on the subgraph. Under no constraints, we simply set \(A^{\prime}=A\). Inference over days is intractable Guruswami et al. (2011). Efficient solutions are known for projective trees Eisner (1996), various classes of mildly nonprojective trees Gomez-Rodriguez (2016), unrestricted spanning trees McDonald et al. (2005), and both unrestricted and weakly connected noncrossing dogs Kuhlmann and Johnsson (2015).

ParsimonySemantic parsers must be able to produce more than projective trees because the share of projective trees is pretty low (under 3%) in semantic graph banks Kuhnlmann and Johnsson (2015). However, if we know that the parses have some restrictions, it is better to use them to restrict the search space as much as possible.

There are two strategies for reducing the search space. One is to develop a specialized inference algorithm for a particular natural language or family of dags, such as weakly connected graphs Kuhlmann and Johnsson (2015). The other strategy is to control the local complexity of digraphs through lexical categories Baldridge and Kruijff (2003) or equivalent mechanisms. This strategy produces a more sensitive model of the language, but requires a principled insight on how the complexity of digraphs can be characterized.

## 3 Constraints on the Search Space

We will now present a classification of digraphs on the basis of their formal properties.

The Noncrossing PropertyFor convenience, graphs and digraphs may be ordered like in a complete digraph of a sentence. Two edges \(\{i,j\}\), \(\{k,l\}\) in an ordered graph or arcs \(i\to j,k\to l\) in an ordered digraph are said to be _crossing_ if \(\min\{i,j\}<\min\{k,l\}<\max\{i,j\}<\max\{k,l\}\). A graph or digraph is _noncrossing_ if it has no crossing edges or arcs. Noncrossing (di)graphs (**NC-(DI)GRAPH**) are the largest possible (di)graphs that can be drawn on a circle without crossing arcs. In the following, we assume that all digraphs and graphs are noncrossing 

An arc \(x\to y\) is _(properly) covered by_ an arc \(z\to t\) if \(\{\{x,y\}\neq\{z,t\}\}\) and min\(\{z,t\}\leq\min\{x,y\}\leq\max\{x,y\}\leq\max\{x,y\}\leq\max\{z,t\}\).

**Ontology** Fig. 1 presents an ontology of such families of loop-free noncrossing digraphs that can be distinguished by digraphs with 5 vertices.

In the digraph ontology, a _multitree_ aka mangrove is a dag with the property of being _strongly unambiguous_ (\(\mathbf{UNAMB_{S}}\)), which asserts that, given two distinct vertices, there is at most one repeat-free path between them [20].1 A _polytree_[37] is a multitree whose underlying graph is a tree. The _out_ property (**OUT**) of a digraph \((V,E)\) means that no vertex \(i\in V\) has two incoming arcs \(\{j,k\}\to i\) s.t. \(j\neq k\).

Footnote 1: A different definition forbids diamonds as minors.

An ordered digraph is _weakly projective_ (\(\mathbf{PROJ_{W}}\)) if for all vertices \(i\), \(j\) and \(k\), if \(k\to j\to i\), then either \(\{i,j\}<k\) or \(\{i,j\}>k\). In other words, the constraint, akin the _outside-to-inside constraint_[46], states that no outgoing arc of a vertex properly covers an incoming arc. This is implied by a stronger constraint known as _Harper, Hays, Lecerf and Ihm projectivity_[9].

We can embed the ontology of graphs (unrestricted, connected, forests and trees) into the ontology of digraphs by viewing an undirected graph \((V,E)\) as an inverse digraph \((V,\{(i,j),(j,i)\mid\{i,j\}\in E\})\). This kind of digraph has an _inverse property_ (**INV**). Its opposite is an _oriented (or.)_ digraph \((V,A)\) where \(i\to j\in A\) implies \(j\to i\notin A\) (defines the property **ORIENTED**). Out forests and trees are, by convention, oriented digraphs with an underlying forest or tree, respectively.

Distinctive PropertiesA few important properties of digraphs are local and can be verified by inspecting each vertex separately with its incident arcs. These include (i) the out property (**OUT**), (ii) the nonstandard projectivity property (\(\mathbf{PROJ_{W}}\)), (iii) the inverse property (**INV**) and (iv) the orientedness (or.) property.

Properties \(\mathbf{UNAMB_{S}}\), \(\mathbf{ACYC_{D}}\), \(\mathbf{CONN_{W}}\), and \(\mathbf{ACYC_{U}}\) are nonlocal properties of digraphs and cannot be generally verified locally, through finite spheres of vertices [8]. The following proposition covers the configurations that we have to detect in order to decide the nonlocal properties of noncrossing digraphs.

**Proposition 1**.: _Let \(G=(V,E)\) be a noncrossing digraph._

* _If_ \(G\notin\mathbf{UNAMB_{S}}\)_, then the digraph contains one of the following four configurations or their reversals:_
* _If_ \(G\notin\mathbf{ACYC_{D}}\)_, then the graph contains one of the configurations_
* _If_ \(G\notin\mathbf{ACYC_{U}}\)_, then the underlying graph contains the following configuration:_
* _If_ \(G\notin\mathbf{CONN_{W}}\)_, then the underlying graph contains one of the following configurations:_

Proposition 1 gives us a means to implement the property tests in practice. It tells us intuitively that although the paths can be arbitrarily long, any underlying cycle containing more than 2 arcs consists of one covering arc and a linear chain of edges between its end points.

## 4 The Set of Digraphs as a Language

In this section, we show that the set of noncrossing digraphs is isomorphic to an unambiguous context-free language over a bracket alphabet.

### Basic Encoding

Any noncrossing ordered graph \(([1,...,n],E)\), even with self-loops, can be encoded as a string of brackets using the algorithm enc in Fig. 2. For example, the output for the ordered graph

Figure 1: Basic properties split the set of 62464 noncrossing digraphs for 5 vertices into 23 classes



for i in [i],..n;j:=1:=1:=1:=1:=s=[] for j in [i-1,...,2,1]; while c in stolin: if (!a) i in E; if e =="; print "]; s.push(n) for j in [n,n-l--,...,i+1]; if c =="];; if (!i,j) in B; i = spob() if (!i,j) in E;

 i in: g.spo() if i,j in E; if c =="{"}; print "|" n = n + 1 if icn;

 printf "{i}" return (n,E)

 is the string [{{}][{}][{}][{}]. Intuitively, pairs of brackets of the form \(\{\}\) can be interpreted as spaces between vertices, and then each set of matching brackets [...] encodes an arc that covers the spaces represented inside the brackets.

Any noncrossing ordered digraph \(([1,\ldots,n],A)\) can be encoded with slight modifications to the algorithm. Instead of printing [ ] for an edge \(\{i,j\}\in E_{A}\), \(i\leq j\), the algorithm should now print

\begin{tabular}{r l}   / & ? & if \((i,j)\in A\),\((j,i)\not\in A\); \\   \(<~{}~{}\backslash\) & if \((i,j)\not\in A\),\((j,i)\in A\); \\   [ ] & if \((i,j),(j,i)\in A\). \\ \end{tabular} In this way, we can simply encode the digraph \((\{1,2,3,4\},\{(1,2),(2,2),(4,1),(4,2)\})\) as the string \(<\langle\{1\}><\{\}|\{\}\setminus\).

**Proposition 2**.: _The encoding respects concatenation where the adjacent nonempty operands have a common vertex._

Context-FreenessArbitrary strings with balanced brackets form a context-free language that is known, generically, as a Dyck language. It is easy to see that the graphs **NC-GRAPH** are encoded with strings that belong to the Dyck language \(D_{2}\) generated by the context-free grammar: \(S\rightarrow\{S\}\,S\mid\{S\}S\mid\varepsilon\). The _encoded graphs_, \(L_{\text{{NC-graph}}}\), are, however, generated exactly by the context-free grammar \(S\rightarrow\{S^{\prime}\}\ S\mid\{\}\ S\mid\varepsilon\), \(S^{\prime}\rightarrow\{S^{\prime}\}\ T\mid\{\}\ S\ T\to\{S^{\prime}\}\ S\mid\{\}\ S\). This language is an _unambiguous_ context-free language.

**Proposition 3**.: _The encoded graphs, \(L_{\text{{NC-GRAPH}}}\), make an unambiguous context-free language._

The practical significance of Proposition 3 is that there is a bijection between \(L_{\text{{NC-GRAPH}}}\) and the derivation trees of a context-free grammar.

### Bracketing Beyond the Encoding Non-Derivational Representation

A non-derivational representation for any context-free language \(L\) has been given by Chomsky and Schutzenberger (1963). This replaces the stack with a Dyck language \(D\) and the grammar rules with co-occurrence patterns specified by a regular language \(Reg\). To hide the internal alphabet from the strings of the represented language, there is a homomorphism that cleans the internal strings of \(Reg\) and \(D\) from internal markup to get actual strings of the target language:

\[L_{\text{{NC-GRAPH}}}=h(D\cap Reg).\]

To make this concrete, replace the previous context free grammar by \(S^{\prime\prime}\rightarrow\{`S^{\prime}\}\ I\ S\mid\{\}\ S\mid\varepsilon\), \(S\rightarrow\{`S^{\prime}\}\ S\mid\{\}\ S\mid\varepsilon\), \(S\rightarrow\{`S^{\prime}\}\ I\ T\mid\{\}\ S\), \(T\rightarrow[\,S^{\prime}\ ]\ S\mid\{\}\ S\). The homomorphism \(h\) (Fig. 3a) would now relate this language to the original language, mapping the string \(\{`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime\{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`\}\}\)\({`\prime{`}\}\)\({`\prime{`\}\}\)\({`\prime{`}\}\)\ 

such a way that the configurations given in Proposition 1 can be detected locally from these.

Classification of Underlying ChainsA _maximal linear chain_ is a maximally long sequence of one or more edges that correspond to an underlying left-to-right path in the underlying graph in such a way that no edge in this chain is properly covered by an edge that does not properly cover all the edges in the chain. For example, the graph contains six maximal linear chains, indicated with their Roman numbers on each arc. We decide non-local properties of noncrossing digraphs by recognizing maximal linear chains as parts of configurations presented in Proposition 1.

Every _loose_ chain (like V and VI) starts with a bracket that is adjacent to a \(\}\)-bracket. Such a chain can contribute only a covering edge to an underlying cycle. In contrast, a bracket with an apostrophe marks the beginning of a _non-loose_ chain that can either start at the first vertex, or share starting point with a covering chain. When a non-loose chain is covered, it can be touched twice by a covering edge. The prefixes of chains are classified incrementally, from left to right, with a finite automaton (Figure 4). All states of the automaton are final and correspond to distinct classes of the chains. These classes are encoded to an extended set of brackets.

The automaton is symmetric: states with uppercase names are symmetrically related with corresponding lowercase states. Thus, it suffices to define the initial and uppercase-named states:

* the initial state for a non-loose chain;
* a bidirectional chain: \(u\leftrightarrow(v\leftrightarrow)y\); A a primarily bidirectional forward chain: \(u\leftrightarrow v\rightarrow y\); F a forward chain: \(u\to v\rightarrow y\); Q a primarily forward chain: \(u\to v\leftrightarrow(\cdots\rightarrow)y\); C a primarily forward 1-turn chain: \(u\to v\leftarrow\gamma\); E a primarily forward 2-turn chain: \(u\rightarrow v\gets x\rightarrow y\); Z a 3-turn chain;
* the initial (and only) state for a loose chain;

Recognition of ambiguous paths in configurations

\[u\overbrace{\longleftarrow\to y}\leftrightarrow\to y\text{ and } u\overbrace{\longleftarrow\to v\to X\leftarrow\to y} \longrightarrow y\text{ involves three chain levels. To support the recognition, subtypes of edges are defined according to the chains they cover. The brackets >I', |I', >I, |I, \lambda, >a, >Q, >Q, >Q, >C, \<E, >e indicate edges that constitute a cycle with the chain they cover. The brackets >V'', >V'', >V'' indicate edges that cover 2-turn chains. Not all states make these distinctions.

Extended RepresentationThe extended brackets encode the latent structure of digraphs: the orientation and the subtype of the edge and the class of the chain. The total alphabet \(\Sigma\) of the strings now contains the boundary brackets \(\{\}\) and 54 pairs of brackets (Figure 4) for edges from which we obtain a new Dyck language, \(D_{55}\), and an extended homomorphism \(h_{\mathrm{lat}}\).

The _Reg_ component of the language representation is replaced with \(Reg_{\mathit{lat}}\), that is, an intersection of (1) an inverse homomorphic image of _Reg_ to strings over the extended alphabet, (2) a local language that constrains adjacent edges according to Figure 4, (3) a local language specifying how the chains start, and (4) a local language that distinguishes pure oriented edges from those that cover a cycle or a 2-turn chain. The new component requires only 24 states as a deterministic automaton.

**Proposition 6**.: \(h_{\mathrm{lat}}(D_{55}\cap Reg_{\mathrm{lat}})\) _is an unambiguous representation for \(L_{\mathbf{NC}\text{-}\mathtt{DGERAPH}}\)._

The internal language \(L_{\mathbf{NC}\text{-}\mathtt{DGERAPH}_{\mathrm{lat}}}=D_{55}\cap Reg_{ \mathrm{lat}}\) is called the set of _latent encoded digraphs_.

ExampleHere is a digraph with its latent encoding:

\[\underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\uparrow$}}_{1} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{2} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{3} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{4} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{5} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{6} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{7} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{7} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{8} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{8} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.0pt}{$\downarrow$}}_{9} \underbrace{<x^{\prime}\hskip-8.0pt\raisebox{-9.

 

(directed path \(1\to 2\to 7\to 1\)), is strongly ambiguous (two directed paths \(2\to 1\) and \(2\to 7\to 1\)) and is not weakly connected (vertices 5 and 6 are not connected to the rest of the digraph).

**Expressing Properties via Forbidden Patterns** We now demonstrate that all the mentioned non-local properties of graphs have become local in the extended internal representation of the code strings \(L_{\mbox{\bf NC-DIGRAPH}}\) for noncrossing digraphs.

These distinctive properties of graph families reduce to forbidden patterns in bracket strings and then compile into regular constraint languages. These are presented in Table 1. To keep the patterns simple, subsets of brackets are defined:

\[\begin{array}{lllll}L_{/}&\left\{\neg,\neg,l\text{brackets}&L_{<}&\left\{ \neg,<\text{brackets}\\ R_{>}&\right\}&\left]\neg,>\text{brackets}&R_{\langle}&\left]\neg,\setminus \text{brackets}\\ B&\left\{,\right\}&R&R_{\rangle}\cup R_{\rangle}\\ R_{\rm loose}&\left\{\right\},\;\succcuit,\;\backslash,\;\backslash,\; \cdot\right\}\,,\;\right]\quad R_{\widetilde{\rm loose}}\quad R-R_{\rm loose} \\ R_{\rm right}&R~{}{\rm reaching}~{}F,Q,I,A&R_{\rm left}&R~{}{\rm reaching}~{}f,q,i,a\\ R_{\rm right2}&>P,\geq,\geq,E,|E&R_{\rm left2}&\backslash p,\geq,\geq,e,\geq e ,\geq e\\ \Sigma_{\rm fin}&L_{c}\cup R_{\varsigma}&\overline{B}&\Sigma-B\\ R_{\rm vergent}&{\rm non\text{--}}^{r}~{}R~{}{\rm reaching}~{}I,\underline{ o},q,A,a,C,\\ \Sigma_{\rm or}&{\rm all~{}brackets~{}for~{}oriented~{}edges}\\ \Sigma_{\rm inv}&{\rm all~{}brackets~{}for~{}inverted~{}edges}\end{array}\]

## 6 Validation Experiments

The current experiments were designed (1) to help in developing the components of \(Reg_{\rm lat}\) and the constraint languages of axiomatic properties, (2) to validate the representation, the constraint languages and their unambiguity, (3) to learn about the ontology and (4) to sample the integer sequences associated with the cardinality of each family in the ontology.

**Finding the Components** Representations of \(Reg_{\rm lat}\) were built with scripts written using a finite-state toolkit Hulden (2009) that supports rapid exploration with regular languages and transducers.

Validation of LanguagesOur scripts presented alternative approaches to compute languages of encoded digraphs with \(n\) vertices up to \(n=9\). We also implemented a Python script that enumerated elements of families of graphs up to \(n=6\). The solutions were used to cross-validate one another.

The constraint \(G_{n}=\overline{B}^{*}(\{\bar{J}\bar{B}^{*}\}^{n-1})\) ensures \(n\)-vertices in encoded digraphs. The finite set of encoded acyclic 5-vertex digraphs was computed with a finite-state approach Yli-Jyra et al. (2012) that takes the input projection of the composition

\[\operatorname{Id}(Reg_{lat}\cap A_{D}\cap G_{5})\circ T_{55}\circ T_{55} \circ T_{55}\circ T_{55}\circ T_{55}\circ\operatorname{Id}(\varepsilon)\]

where Id defines an identity relation and transducer \(T_{55}\) eliminates matching adjacent brackets. This composition differs from the typical use where the purpose is to construct a regular relation Kaplan and Kay (1994) or its output projection Roche (1996); Offazer (2003).

For digraphs with a lot of vertices, we had an option to employ a dynamic programming scheme Yli-Jyra (2012) that uses weighted transducers.

Building the OntologyTo build the ontology in Figure 1 we first found out which combinations of digraph properties co-occur to define distinguishable families of digraphs. After the nodes of the

\begin{table}
\begin{tabular}{|p{85.4pt} c l|} \hline  & & Property & Constraint language \\ \hline \multirow{2}{*}{\(K_{\rm locse}(no connecting edges)\) (a vertex without edges)} & \multirow{2}{*}{\(\mathbf{ACYC}_{\rm U}\)} & \multirow{2}{*}{\(A_{U}=\Sigma^{*}-\Sigma^{*}R_{\rm loose}R\Sigma^{*}\)} \\  & & & \\   & & \(\mathbf{CONN}_{\rm W}\) & \\   & & \(\mathbf{ACYC}_{\rm D}\) & \\   & & \(\mathbf{UNAMB}_{\rm S}\) & \\   & & \(\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad\qquad \qquad 

lattice were found, we were able to see the partial order between these.

Integer SequencesWe sampled, for important families of digraphs, the prefixes of their related integer sequences. We found out that each family of graphs is pretty much described by its cardinality, see Table 2. In many cases, the number sequence was already well documented OEIS Foundation Inc., 2017.

## 7 The Formal Basis of Practical Parsing

While presenting a practical parser implementation is outside of the scope of this paper, which focuses in the theory, we outline in this section the aspects to take into account when applying our representation to build practical natural language parsers.

Positioned BracketsIn order to do inference in arc-factored parsing, we incorporate weights to the representation. For each vertex in \(G_{n}\), the brackets are decorated with the respective position number. Then, we define an input-specific grammar representation where each pair of brackets in \(D\) gets an arc-factored weight given the directions and the vertex numbers associated with the brackets.

Grammar IntersectionWe associate, to each \(G_{n}\), a quadratic-size context-free grammar that generates all noncrossing digraphs with \(n\) vertices. This grammar is obtained by computing (or even precomputing) the intersection \(D_{55}\cap Reg_{\text{lat}}\cap G_{n}\) in any order, exploiting the closure of context-free languages under intersection with regular languages Bar-Hillel et al. (1961). The introduction of the position numbers and weights in the Dyck language gives us, instead, a weighted grammar and its intersection Lang (1994). This grammar is a compact representation for a finite set of weighted latent encoded digraphs. Additional constraints during the intersection tailors the grammar to different families of digraphs.

Dynamic ProgrammingThe heaviest digraph is found with a dynamic programming algorithm that computes, for each nonterminal in the grammar, the weight of the heaviest subtree. A careful reader may notice some connections to Eisner algorithm Eisner and Satta (1999), context-free parsing through intersection Nederhof and Satta (2003), and a dynamic programming scheme that uses contracting transducers and factorized composition Yli-Jyra (2012). Unfortunately, space does not permit discussing the connections here.

Lexicalized Search SpaceIn practical parsing, we want the parser behavior and the dependency structure to be sensitive to the lexical entries or features of each word. We can replace the generic vertex description \(\overline{B}^{*}\) in \(G_{n}\) with subsets that depend on respective lexical entries. Graphical constraints can be applied to some vertices but relaxed for others. This application of current results gives a principled, graphically motivated solution to lexicalized control over the search space.

## 8 Conclusion

We have investigated the search space of parsers that produce noncrossing digraphs. Parsers that can be adapted to different needs are less dependent on artificial assumptions on the search space. Adaptivity gives us freedom to model how the properties of digraphs are actually distributed in linguistic data. As the adaptive data analysis deserves to be treated in its own right, the current work focuses on the separation of the parsing algorithm from the properties of the search space.

This paper makes four significant contributions.

Contribution 1: Digraph EncodingThe paper introduces, for noncrossing digraphs, an encoding that uses brackets to indicate edges.

Bracketed trees are widely used in generative syntax, treebanks and structured document formats. There are established conversions between phrase structure and projective dependency trees, but the currently advocated edge bracketing is expressive and captures more than just projective dependency trees. This capacity is welcome as syntactic and semantic analysis with dependency graphs is a steadily growing field.

The edge bracketing creates new avenues for the study of connections between noncrossing graphs and context-free languages, as well as their recognizable properties. By demonstrating that digraphs can be treated as strings, we suggest that practical parsing to these structures could be implemented with existing methods that restrict context-free grammars to a regular yield language.

Contribution 2: Context-Free PropertiesAcyclicity and other important properties of noncrossing digraphs are expressible as unambiguous context-free sets of encoded noncrossing 

digraphs. This facilitates the incorporation of property testing to dynamic programming algorithms that implement exact inference.

Descriptive complexity helps us understand to which degree various graphical properties are local and could be incorporated into efficient dynamic programming during exact inference. It is well known that acyclicity and connectivity are not definable in first-order logic (FO) while they can be defined easily in monadic second order logic (MSO) Courcelle (1997). MSO involves set-valued variables whose use in dynamic programming algorithms and tabular parsing is inefficient. MSO queries have a brute force transformation to first-order (FO) logic, but this does not generally help either as it is well known that MSO can express intractable problems.

The interesting observation of the current work is that some MSO definable properties of digraphs become local in our extended encoding. This encoding is linear compared to the size of digraphs: each string over the extended bracket alphabet encodes a fixed assignment of MSO variables. The properties of noncrossing digraphs now reduce to properties of bracketed trees with linear amount of latent information that is fixed for each digraph.

A deeper explanation for our observation comes from the fact that the treewidth of noncrossing and other outerplanar graphs is bounded to 2. When the treewidth is bounded, all MSO definable properties, including the intractable ones, become linear time decidable for individual structures Courcelle (1990). They can also be decided in a logarithmic amount of writable space Elberfeld et al. (2010), e.g. with element indices instead of sets. By combining this insight with Proposition 1, we obtain a logspace solution for testing acyclicity of a noncrossing graph (Figure 5).

Although bounded treewidth is a weaker constraint than so-called bounded treedepth that would immediately guarantee first-order definabil

\begin{table}
\begin{tabular}{|l l|c c|} \hline Name & Sequence prefix for \(n=2,3,...\) & Example \\ \hline digraph & **(KJ)** ; 46.4179; **62.64842** ; 437120,101859328 & weakly projective & 4.3680; **77.4138880** ; 2661376 & \\  & _RwC_ (_P_ (_G_ (_R_ (_ (_ (_ (_ (_ (_ (_ (_ (_  (_ 

ity (Elberfeld et al., 2016), it can sometimes turn intractable search problems to dynamic programming algorithms (Akutsu and Tamura, 2012). In our case, Proposition 1 gave rise to unambiguous context-free subsets of \(L_{\text{NC-DIGraph}}\). These can be recognized with dynamic programming and used in efficient constrained inference when we add vertex indices to the brackets and weights to the grammar of the corresponding Dyck language.

Contribution 3: Digraph OntologyThe context-free properties of encoded digraphs have elegant nonderivative language representations and they generate a semi-lattice under language intersection. Although context-free languages are not generally closed under intersection, all combinations of the properties in this lattice are context-free and define natural families of digraphs. The nonderivative representations for our axiomatic properties share the same Dyck language \(D_{55}\) and homomorphism, but differ in terms of forbidden patterns. As a consequence, also any conjunctive combination of these two properties shares these components and thus define a context-free language. The obtained semilattice is an ontology of families of noncrossing digraphs.

Our ontology contains important families of noncrossing digraphs used in syntactic and semantic dependency parsing: out trees, dags, and weakly connected digraphs. It shows the entailment between the properties and proves the existence of less known families of noncrossing digraphs such as strongly unambiguous digraphs and oriented graphs, multitrees, oriented forests and polytrees. These are generalizations of out oriented trees. However, these families can still be weakly projective. Table 2 shows integer sequences obtained by enumerating digraphs in each family. At least twelve of these sequences are previously known, which indicates that the families are natural.

We used a finite-state toolkit to build the components of the nongenerative language representation for latent encoded digraphs and the axioms.2

Footnote 2: The finite-state toolkit scripts and a Python-based graph enumerator are available at [https://github.com/amikael/ncdigraphs](https://github.com/amikael/ncdigraphs).

Contribution 4: Generic ParsingThe fourth contribution of this paper is to show that parsing algorithms can be separated from the formal properties of their search space.

All the presented families of digraphs can be treated by parsers and other algorithms (e.g. enumeration algorithms) in a uniform manner. The parser's inference rules can stay constant and the choice of the search space is made by altering the regular component of the language representation.

The ontology of the search space can be combined with a constraint relaxation strategy, for example, when an out tree is a preferred analysis, but a dag is also possible as an analysis when no tree is strong enough. The flexibility applies also to dynamic programming algorithms that complement with the encoding and allow inference of best dependency graphs in a family simply by intersection with a weighted CFG grammar for a Dyck language that models position-indexed edges of the complete digraph.

Since the families of digraphs are distinguished by forbidden local patterns, the choice of search space can be made purely on lexical grounds, blending well with lexicalized parsing and allowing possibilities such as choosing, per each word, what kind of structures the word can go with.

Future workWe are planning to extend the coverage of the approach by exploring 1-endpoint-crossing and \(MH_{k}\) trees (Pitler et al., 2013; Gomez-Rodriguez, 2016), and related digraphs -- see (Yli-Jyra, 2004; Gomez-Rodriguez et al., 2011). Properties such as _weakly projective_, _out_, and _strongly unambiguous_ prompt further study.

An interesting avenue for future work is to explore higher order factorizations for noncrossing digraphs and the related inference. We would also like to have more insight on the transformation of MSO definable properties to the current framework and to logspace algorithms.

## Acknowledgements

AYJ has received funding as Research Fellow from the Academy of Finland (dec. No 270354 - A Usable Finite-State Model for Adequate Syntactic Complexity) and Clare Hall Fellow from the University of Helsinki (dec. RP 137/2013). CGR has received funding from the European Research Council (ERC) under the European Union's Horizon 2020 research and innovation programme (grant agreement No 714150 - FASTPARSE) and from the TELEPARES-UDC project (FFI2014-51978-C2-2-R) from MINECO. The comments of Juha Kontinen, Mark-Jan Nederhof and the anonymous reviewers helped to improve the paper.

 



## References

* Akutsu and Tamura (2012) Tatsuya Akutsu and Takeyuki Tamura. 2012. A polynomial-time algorithm for computing the maximum common subgraph of outerplanar graphs of bounded degree. In Branislav Rovan, Vladimiro Sassone, and Peter Widmayer, editors, _Mathematical Foundations of Computer Science 2012: 37th International Symposium, MFCS 2012, Bratislava, Slovakia, August 27-31, 2012. Proceedings_, Springer Berlin Heidelberg, Berlin, Heidelberg, pages 76-87. [https://doi.org/10.1007/978-3-642-32589-2_10](https://doi.org/10.1007/978-3-642-32589-2_10).
* Volume 1_. Association for Computational Linguistics, Budapest, Hungary, pages 211-218. [https://doi.org/10.3115/1067807.1067836](https://doi.org/10.3115/1067807.1067836).
* Bar-Hilleel et al. (1961) Yehoshua Bar-Hilleel, Micha Perles, and Eliahu Shamir. 1961. On formal properties of simple phrase structure grammars. _Zeitschrift fur Phonologie, Sprachwissenschaft und Kommunikationsforschung_ 14:113-124.
* Chomsky and Schutzenberger (1963) Noam Chomsky and Marcel-Paul Schutzenberger. 1963. The algebraic theory of context-free languages. _Computer Programming and Formal Systems_ pages 118-161.
* 75. [https://doi.org/10.1016/0890-5401](https://doi.org/10.1016/0890-5401)(90)90043-H.
* Courcelle (1997) Bruno Courcelle. 1997. The expression of graph properties and graph transformations in monadic second-order logic. In G. Rozenberg, editor, _Handbook of Graph Grammars and Computing by Graph Transformations_, World Scientific, New-Jersey, London, volume 1, chapter 5, pages 313-400.
* Esner (1996) Jason Eisner. 1996. Three new probabilistic models for dependency parsing: An exploration. In _Proceedings of the 16th International Conference on Computational Linguistics (COLING-96)_. Copenhagen, Denmark, pages 340-345. [http://aclweb.org/anthology/C/C96/C96-1058.pdf](http://aclweb.org/anthology/C/C96/C96-1058.pdf).
* Esner and Satta (1999) Jason Eisner and Giorgio Satta. 1999. Efficient parsing for bilexical context-free grammars and Head Automaton Grammars. In _Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics_. Association for Computational Linguistics, College Park, Maryland, USA, pages 457-464. [https://doi.org/10.3115/1034678.1034748](https://doi.org/10.3115/1034678.1034748).
* Elberfeld et al. (2016) Michael Elberfeld, Martin Grohe, and Till Tantau. 2016. Where first-order and monadic second-order logic coincide. _ACM Trans. Comput. Logic_ 17(4):25:1-25:18. [https://doi.org/10.1145/2946799](https://doi.org/10.1145/2946799).
* Elberfeld et al. (2010) Michael Elberfeld, Andreas Jakoby, and Till Tantau. 2010. Logspace versions of the theorems of Bodlaender and Courcelle. In _Proceedings of the 2010 IEEE 51st Annual Symposium on Foundations of Computer Science_. IEEE Computer Society, Washington, DC, USA, FOCS '10, pages 143-152. [https://doi.org/10.1109/FOCS.2010.21](https://doi.org/10.1109/FOCS.2010.21).
* Gomez-Rodriguez (2016) Carlos Gomez-Rodriguez. 2016. Restricted non-projectivity: Coverage vs. efficiency. _Computational Linguistics_ 42(4):809-817. [https://doi.org/10.1162/COLI.a.00267](https://doi.org/10.1162/COLI.a.00267).
* Gomez-Rodriguez et al. (2011) Carlos Gomez-Rodriguez, John A. Carroll, and David J. Weir. 2011. Dependency parsing schemata and mildly non-projective dependency parsing. _Computational Linguistics_ 37(3):541-586. [https://doi.org/10.1162/COLI.a..00060](https://doi.org/10.1162/COLI.a..00060).
* Gradel et al. (2005) Erich Gradel, P. G. Kolaitis, L. Libkin, M. Marx, J. Spencer, Moshe Y. Vardi, Y. Venema, and Scott Weinstein. 2005. _Finite Model Theory and Its Applications (Texts in Theoretical Computer Science. An EATCS Series)_. Springer-Verlag New York, Inc., Secaucus, NJ, USA.
* Greibach (1973) Sheila Greibach. 1973. The hardest context-free language. _SIAM Journal on Computing_ 2(4):304-310. [https://doi.org/10.1137/0202025](https://doi.org/10.1137/0202025).
* Guruswami et al. (2011) Venkatesan Guruswami, Johan Hastad, Rajsekar Manokaran, Prasad Raghavendra, and Moses Charikar. 2011. Beating the random ordering is hard: Every ordering CSP is approximation resistant. _SIAM Journal on Computing_ 40(3):878-914. [https://doi.org/10.1137/090756144](https://doi.org/10.1137/090756144).
* Holden (2009) Mans Hulden. 2009. Foma: a finite-state compiler and library. In _Proceedings of the Demonstrations Session at EACL 2009_. Association for Computational Linguistics, Athens, Greece, pages 29-32. [http://www.aclweb.org/anthology/E09-2008](http://www.aclweb.org/anthology/E09-2008).
* Hudden (2011) Mans Hulden. 2011. Parsing CFGs and PCFGs with a Chomsky-Schutzenberger representation. In Zygmund Vetulauriu, editor, _Human Language Technology. Challenges for Computer Science and Linguistics: 4th Language and Technology Conference, LTC 2009, Poznan, Poland, November 6-8, 2009, Revised Selected Papers_, Springer Berlin Heidelberg, Berlin, Heidelberg, pages 151-160. [https://doi.org/10.1007/978-3-642-20095-3_14](https://doi.org/10.1007/978-3-642-20095-3_14).
* Kaplan and Kay (1994) Ronald M. Kaplan and Martin Kay. 1994. Regular models of phonological rule systems. _Computational Linguistics_ 20(3):331-378. [http://dl.acm.org/citation.cfm?id=204915.204917](http://dl.acm.org/citation.cfm?id=204915.204917).
* Kuhlmann (2015) Marco Kuhlmann. 2015. Tabulation of non-crossing acyclic digraphs. arXiv:1504.04993. [https://arxiv.org/abs/1504.04993](https://arxiv.org/abs/1504.04993).
* Kuhlmann and Johnsson (2015) Marco Kuhlmann and Peter Johnsson. 2015. Parsing to noncrossing dependency graphs. _Transactions of the Association for Computational Linguistics_ 3:559-570. [http://aclweb.org/anthology/Q/Q15/Q15-1040.pdf](http://aclweb.org/anthology/Q/Q15/Q15-1040.pdf)

Marco Kuhlmann and Stephan Oepen. 2016. Towards a catalogue of linguistic graph banks. _Computational Linguistics_ 42(4):819-827. [https://doi.org/10.1162/COLI.a..00268](https://doi.org/10.1162/COLI.a..00268).
* Lang [1994] Bernard Lang. 1994. Recognition can be harder than parsing. _Computational Intelligence_ 10(4):486-494. [http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8640.1994.tb00011.x/full](http://onlinelibrary.wiley.com/doi/10.1111/j.1467-8640.1994.tb00011.x/full).
* Lange [1997] Klaus-Jorn Lange. 1997. An unambiguous class possessing a complete set. In Morvan Reischuk, editor, _STACKS'97 Proceedings_. Springer, volume 1200 of _Lecture Notes in Computer Science_. [http://dl.acm.org/citation.cfm?id=695352](http://dl.acm.org/citation.cfm?id=695352).
* Marcus [1967] S. Marcus. 1967. _Algebraic Linguistics; Analytical Models_, volume 29 of _Mathematics in Science and Engineering_. Academic Press, New York and London.
* McDonald et al. [2005] Ryan McDonald, Fernando Pereira, Kiril Ribarov, and Jan Hajic. 2005. Non-projective dependency parsing using spanning tree algorithms. In _Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing_. Association for Computational Linguistics, Vancouver, British Columbia, Canada, pages 523-530. [http://www.aclweb.org/anthology/H/H05/H05-1066.pdf](http://www.aclweb.org/anthology/H/H05/H05-1066.pdf).
* Nederhof and Satta [2003] Mark-Jan Nederhof and Giorgio Satta. 2003. Probabilistic parsing as intersection. In _8th International Workshop on Parsing Technologies_. LORIA, Nancy, France, pages 137-148.
* Inc [2017] OEIS Foundation Inc. 2017. The on-line encyclopedia of integer sequences. [http://oeis.org](http://oeis.org), read on 15 January 2017.
* Oepen et al. [2015] Stephan Oepen, Marco Kuhlmann, Yusuke Miyao, Daniel Zeman, Silvie Cinkova, Dan Flickinger, Jan Hajic, and Zdenka Uresova. 2015. Semeval 2015 task 18: Broad-coverage semantic dependency parsing. In _Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015)_. Association for Computational Linguistics, Denver, Colorado, pages 915-926. [http://www.aclweb.org/anthology/S15-2153](http://www.aclweb.org/anthology/S15-2153).
* Oflazer [2003] Kemal Oflazer. 2003. Dependency parsing with an extended finite-state approach. _Computational Linguistics_ 29(4):515-544. [https://doi.org/10.1162/089120103322753338](https://doi.org/10.1162/089120103322753338).
* Pitler et al. [2013] Emily Pitler, Sampath Kannan, and Mitchell Marcus. 2013. Finding optimal 1-endpoint-crossing trees. _Transactions of the Association for Computational Linguistics_ 1:13-24. [http://aclweb.org/anthology/Q13-1002](http://aclweb.org/anthology/Q13-1002).
* Rebane and Pearl [1987] George Rebane and Judeau Pearl. 1987. The recovery of causal poly-trees from statistical data. In _Proceedings of the 3rd Annual Conference on Uncertainty in Artificial Intelligence (UAI 1987)_. Seattle, WA, pages 222-228. [http://dl.acm.org/citation.cfm?id=3023784](http://dl.acm.org/citation.cfm?id=3023784).
* Roche [1996] Emmanuel Roche. 1996. Transducer parsing of free and frozen sentences. _Natural Language Engineering_ 2(4):345-350. [https://doi.org/10.1017/S1351324997001605](https://doi.org/10.1017/S1351324997001605).
* Schluter [2014] Natalie Schluter. 2014. On maximum spanning DAG algorithms for semantic DAG parsing. In _Proceedings of the ACL 2014 Workshop on Semantic Parsing_. Association for Computational Linguistics, Baltimore, MD, pages 61-65. [http://www.aclweb.org/anthology/W/W14/W14-2412.pdf](http://www.aclweb.org/anthology/W/W14/W14-2412.pdf).
* Schluter [2015] Natalie Schluter. 2015. The complexity of finding the maximum spanning DAG and other restrictions for DAG parsing of natural language. In _Proceedings of the Fourth Joint Conference on Lexical and Computational Semantics_. Association for Computational Linguistics, Denver, Colorado, pages 259-268. [http://www.aclweb.org/anthology/S15-1031](http://www.aclweb.org/anthology/S15-1031).
* Yli-Jyra [2004] Anssi Yli-Jyra. 2004. Axiomatization of restricted non-projective dependency trees through finite-state constraints that analyse crossing bracketings. In Geert-Jan M. Kruijff and Denys Ducher, editors, _COLING 2004 Recent Advances in Dependency Grammar_. COLING, Geneva, Switzerland, pages 25-32. [https://www.aclweb.org/anthology/W/W04/W04-1504.pdf](https://www.aclweb.org/anthology/W/W04/W04-1504.pdf).
* Yli-Jyra [2005] Anssi Yli-Jyra. 2005. Approximating dependency grammars through intersection of star-free regular languages. _Int. J. Found. Comput. Sci._ 16(3):565-579. [https://doi.org/10.1142/S0129054105003169](https://doi.org/10.1142/S0129054105003169).
* Yli-Jyra [2012] Anssi Yli-Jyra. 2012. On dependency analysis via contractions and weighted FSTs. In Diana Santos, Krister Linden, and Wanjiku Ng'ang'a, editors, _Shall We Play the Festschrift Game?, Essays on the Occasion of Lauri Carlson's 60th Birthday_. Springer, pages 133-158. [https://doi.org/10.1007/978-3-642-30773-7_10](https://doi.org/10.1007/978-3-642-30773-7_10).
* Yli-Jyra et al. [2012] Anssi Yli-Jyra, Jussi Piitulainen, and Atro Voutilainen. 2012. Refining the design of a contracting finite-state dependency parser. In Iinaki Alegria and Mans Hulden, editors, _Proceedings of the 10th International Workshop on Finite State Methods and Natural Language Processing_. Association for Computational Linguistics, Donostia-San Sebastian, Spain, pages 108-115. [http://www.aclweb.org/anthology/W12-6218](http://www.aclweb.org/anthology/W12-6218).
* Zhang and Nivre [2011] Yue Zhang and Joakim Nivre. 2011. Transition-based dependency parsing with rich non-local features. In _Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies_. Association for Computational Linguistics, Portland, Oregon, USA, pages 188-193. [http://www.aclweb.org/anthology/P11-2033](http://www.aclweb.org/anthology/P11-2033)