We present an inexact search algorithm for the problem of predicting a two-layered dependency graph . The algorithm is based on a k-best version of the standard cubictime search algorithm for projective dependency parsing , which is used as the backbone of a beam search procedure . This allows us to handle the complex nonlocal feature dependencies occurring in bistratal parsing if we model the interdependency between the two layers . We apply the algorithm to the syntacticsemantic dependency parsing task of the CoNLL-2008 Shared Task , and we obtain a competitive result equal to the highest published for a system that jointly learns syntactic and semantic structure . Numerous linguistic theories assume a multistratal model of linguistic structure , such as a layer of surface syntax , deep syntax , and shallow semantics . Examples include Meaning-Text Theory ( Mel'čuk , 1988 ) , Discontinuous Grammar ( Buch-Kromann , 2006 ) , Extensible Dependency Grammar ( Debusmann et al . , 2004 ) , and the Functional Generative Description ( Sgall et al . , 1986 ) which forms the theoretical foundation of the Prague Dependency Treebank ( Hajič , 1998 ) . In the statistical NLP community , the most widely used grammatical resource is the Penn Treebank ( Marcus et al . , 1993 ) . This is a purely syntactic resource , but we can also include this treebank in the category of multistratal resources since the PropBank ( Palmer et al . , 2005 ) and NomBank ( Meyers et al . , 2004 ) projects have annotated shallow semantic structures on top of it . Dependency-converted versions of the Penn Treebank , PropBank and NomBank were used in the CoNLL-2008 Shared Task ( Surdeanu et al . , 2008 ) , in which the task of the participants was to produce a bistratal dependency structure consisting of surface syntax and shallow semantics . Producing a consistent multistratal structure is a conceptually and computationally complex task , and most previous methods have employed a purely pipeline-based decomposition of the task . This includes the majority of work on shallow semantic analysis ( Gildea and Jurafsky , 2002 , inter alia ) . Nevertheless , since it is obvious that syntax and semantics are highly interdependent , it has repeatedly been suggested that the problems of syntactic and semantic analysis should be carried out simultaneously rather than in a pipeline , and that modeling the interdependency between syntax and semantics would improve the quality of all the substructures . The purpose of the CoNLL-2008 Shared Task was to study the feasibility of a joint analysis of syntax and semantics , and while most participating systems used a pipeline-based approach to the problem , there were a number of contributions that attempted to take the interdependence between syntax and semantics into account . The top-performing system in the task ( Johansson and Nugues , 2008 ) applied a very simple reranking scheme by means of a k-best syntactic output , similar to previous attempts ( Gildea and Jurafsky , 2002 ; Toutanova et al . , 2005 ) to improve semantic role labeling performance by using mul-tiple parses . The system by Henderson et al . ( 2008 ) extended previous stack-based algorithms for dependency parsing by using two separate stacks to build the syntactic and semantic graphs . Lluís and Màrquez ( 2008 ) proposed a model that simultaneously predicts syntactic and semantic links , but since its search algorithm could not take the syntactic-semantic interdependencies into account , a pre-parsing step was still needed . In addition , before the CoNLL-2008 shared task there have been a few attempts to jointly learn syntactic and semantic structure ; for instance , Merlo and Musillo ( 2008 ) appended semantic role labels to the phrase tags in a constituent treebank and applied a conventional constituent parser to predict constituent structure and semantic roles . In this paper , we propose a new approximate search method for bistratal dependency analysis . The search method is based on a beam search procedure that extends a k-best version of the standard cubic-time search algorithm for projective dependency parsing . This is similar to the search method for constituent parsing used by Huang ( 2008 ) , who referred to it as cube pruning , inspired by an idea from machine translation decoding ( Chiang , 2007 ) . The cube pruning approach , which is normally used to solve the arg max problem , was also recently extended to summing problems , which is needed in some learning algorithms ( Gimpel and Smith , 2009 ) . We apply the algorithm on the CoNLL-2008 Shared Task data , and obtain the same evaluation score as the best previously published system that simultaneously learns syntactic and semantic structure ( Titov et al . , 2009 ) . In this paper , we have presented a new approximate search method to solve the problem of jointly predicting the two layers in a bistratal dependency graph . The algorithm shows competitive performance on the treebank used in the CoNLL-2008 Shared Task , a bistratal treebank consisting of a surface-syntactic and a shallow semantic layer . In addition to the syntactic-semantic task that we have described in this paper , we believe that our method can be used in other types of multistratal syntactic frameworks , such as a representation of surface and deep syntax as in Meaning-Text Theory ( Mel'čuk , 1988 ) . The optimization problem that we set out to solve is intractable , but we have shown that reasonable performance can be achieved with an inexact , beam search-based search method . This is not obvious : it has previously been shown that using an inexact search procedure when the learning algorithm assumes that the search is exact may lead to slow convergence or even divergence ( Kulesza and Pereira , 2008 ) , but this does not seem to be a problem in our case . While we used a beam search method as the method of approximation , other methods are certainly possible . An interesting example is the recent system by Smith and Eisner ( 2008 ) , which used loopy belief propagation in a dependency parser using highly complex features , while still maintaining cubic-time search complexity . An obvious drawback of our approach compared to traditional pipeline-based semantic role labeling methods is that the speed of the algorithm is highly dependent on the size of the interdependency feature representation Φ i . Also , extracting these features is fairly complex , and it is of critical importance to implement the feature extraction procedure efficiently since it is one of the bottlenecks of the algorithm . It is plausible that our performance suffers from the absence of other frequently used syntax-based features such as dependent-of-dependent and voice . It is thus highly dubious that a joint modeling of syntactic and semantic structure is worth the additional implementational effort . So far , no system using tightly integrated syntactic and semantic processing has been competitive with the best systems , which have been either completely pipelinebased ( Che et al . , 2008 ; Ciaramita et al . , 2008 ) or employed only a loose syntactic-semantic coupling ( Johansson and Nugues , 2008 ) . It has been conjectured that modeling the semantics of the sentence would also help in syntactic disambiguation ; however , it is likely that this is already implicitly taken into account by the lexical features present in virtually all modern parsers . In addition , a problem that our beam search method has in common with the constituent parsing method by Huang ( 2008 ) is that highly nonlocal features must be computed late . In our case , this means that if there is a long distance between a predicate and an argument , the secondary link between them will be unlikely to influence the final search result .

Challenge: While it is obvious that syntax and semantics are interdependent, most works focus on developing pipeline-based approaches, hindering quality improvements.
Approach: They propose an inexact search algorithm that predicts jointly the two layers in a bistratal dependency graph based on a beam search procedure.
Outcome: The proposed algorithm performs comparably to the best system on the treebank from CoNLL-2008 Shared Task by learning syntactic and semantics jointly.