Probing the nature of an island constraint with a parsed corpus (2019)

Abstract: This paper presents a case study of the use of the NINJAL Parsed Corpus of Modern Japanese (NPCMJ) for syntactic research. NPCMJ is the first phrase structure-based treebank for Japanese that is specifically designed for application in linguistic (in addition to NLP) research. After discussing some basic methodological issues pertaining to the use of treebanks for theoretical linguistics research, we introduce our case study on the status of the Coordinate Structure Constraint (CSC) in Japanese, showing that NPCMJ enables us to easily retrieve examples that support one of the key claims of Kubota and Lee (2015): that the CSC should be viewed as a pragmatic, rather than a syntactic constraint. The corpus-based study we conducted moreover revealed a previously unnoticed tendency that was highly relevant for further clarifying the principles governing the empirical data in question. We conclude the paper by briefly discussing some further methodological issues brought up by our case study pertaining to the relationship between linguistic research and corpus development.

## introduction this paper presents a case study of applying the ninjal parsed corpus of modern japanese (npcmj; http://npcmj.ninjal.ac.jp/ ) for syntactic research. specifically, we explore the possibility of using npcmj for collecting attested data corroborating the claim made by @xcite (k&l) on the status of the coordinate structure constraint (csc) in japanese. the results are positive. npcmj proved to be an effective tool for extracting sentences that crucially corroborate the claim by k&l that csc should not be viewed as a syntactic constraint. the treebank search moreover identified a tendency that is arguably difficult to find by alternative methods (such as introspective judgments and unannotated or lightly annotated linguistic data), but which is relevant for further clarifying the principles governing the empirical data in question. with these results, we hope to convince the reader that treebanks are highly effective tools for addressing questions that have direct theoretical relevance. the present paper also discusses some possible challenges for using treebanks for linguistic research. we include this discussion since we believe that one can exploit the full power of treebanks only by having an accurate knowledge of what they are, and this is why it is important, on the part of users of treebanks, to understand at least some of the key issues involved in the construction of treebanks. 1 the paper is structured as follows. we start with a discussion of some methodological and practical issues pertaining to the use of treebanks in linguistic research, so as to situate the present case study within a larger context. we then present our case study on the csc patterns in japanese. the case study reveals some further methodological issues pertaining to the relationship between treebank development and linguistic research. we conclude the paper by briefly commenting on these larger issues. ## some methodological issues treebanks are corpora with full syntactic annotation. this means that they are special type of corpora that come with unusually fine-grained linguistic/grammatical information as compared to ordinary large-scale corpora. for this reason, even if one is primarily interested in just using some existing treebank for one's research and not in actually developing such a corpus, it is still instructive to learn about the general challenges 1 previous studies employing treebanks for linguistic research are still surprisingly few in number despite the fact that treebanks for at least major european languages have been around for quite a long time by now. to get a feel for some representative studies, see @xcite , which contains a concise summary of the studies by @xcite and @xcite . the proceedings of the conference 'treebanks and linguistic theories' contain many papers exploiting the use of treebanks for addressing various issues in linguistic research. in our own experience, the following issues stood out as potential bottlenecks in using npcmj as a tool for linguistic research: these issues essentially all pertain to the inherent nature of treebanks: by their very nature, treebanks embody the tension between 'bottomup' descriptive archiving of linguistic data and 'top-down' linguistic theorizing guiding us what kinds of information to annotate. in the rest of this section, we discuss these issues briefly. ## case study: a corpus-based study of the status of the coordinate structure constraint in japanese in this section, we report on a case study we have conducted on the status of the so-called 'coordinate structure constraint' (csc) in japanese using the npcmj corpus. specifically, we were interested in the question of whether corpus data can be used to either validate or invalidate the claim made by @xcite that the csc should be viewed as a pragmatic principle rather than a syntactic constraint. to state the conclusion first, despite some limitations due to the methodological issues discussed in the previous section, the overall results were positive in the following three respects: first, we were able to find occurrences of sentences in the corpus that crucially support k&l's key argument against syntactic accounts of the csc; second, we were able to additionally identify a previously overlooked but important tendency in the data; third, this newly discovered tendency was one which was arguably difficult to identify without the use of a treebank. to make the present paper self-contained, we first review the relevant theoretical literature, both on the csc (section 3.1) and on the relevant facts we focused on in japanese (section 3.2). foot_5 ## some further methodological issues we now turn to some further methodological issues that the present case study brings up, pertaining to the relationship between linguistic theory and corpus development. the ideal relationship between the two is one in which they inform each other. this is easier said than done. in practice, the distance between the thinking behind theoretical linguistics and that behind corpus development is often a source of confusion. the majority of treebank users are not themselves developers of the resources they use, and in this situation, figuring out the exact relationship between the annotation reflected in the corpus and one's own theoretical persuasion tends to be challenging. the most important source of information is of course the annotation manual. in consulting the annotation manual, it is essential to keep in mind the overall design policy of the corpus, which is sometimes explicitly stated, but sometimes only implicit. in the case of the npcmj corpus, we found it useful to distinguish two major factors that guide the current annotation policy, in order to make better sense of the particular treatments of specific linguistic phenomena. one of these is the principle explicitly noted in the annotation manual that the annotation aims to achieve descriptive adequacy and maximum convenience in the retrieval of information. the other, less obvious factor is the fact that the npcmj corpus (or, more precisely, the keyaki treebank that it is based on) was originally designed to serve as a learning model for the syntactic component of an integrated system of syntactic/semantic parser @xcite . the policy to distinguish 'gapped' and 'gapless' noun-modifying clauses can be seen as reflecting the first annotation policy, since this distinction roughly corresponds to the well-known distinction between the 'inner @xcite due to @xcite influential work in descriptive grammar. it should be pointed out here that this annotation policy served our purpose of identifying 'csc violation' noun-modifying clauses despite the fact that, on the face of it, it was directly at odds with our own theoretical position (where, following @xcite , we take all noun-modifying clauses to be gapless). one important lesson to draw from this is that when using a corpus as a tool for linguistic research, one should not confuse annotation policies with theoretical claims. this, however, raises a related, and quite complex issue of whether a completely theory-neutral treebank development is possible. a short (and perhaps obvious) answer to this question is that, in practice, some kind of theoretical insight needs to be 'behind the scene', as it were, guiding all practical decisions in setting up the treebank annotation scheme. and here, the second, 'hidden' guiding principle of npcmj becomes relevant. again, taking the distinction between 'gapped' and 'gapless' relative clauses as an example, the policy to explicitly distinguish these two types of noun-modifying constructions finds support not only from convenience for search but also (at least partly) from the need to use the syntactic annotation as a basis for building semantic representations. if we take this other consideration seriously, there is a sense in which this annotation policy (at least implicitly) reflects a particular theoretical/analytic stance regarding the architecture of the syntax/semantics/pragmatics interface. to the extent that this is a theoretical claim, it contradicts the theoretical position of k&l (and of the authors of the present paper). this is so because it is a claim that the distinction between 'gapped' and 'gapless' noun-modifying clauses needs to be encoded at the level of syntax in order to build adequate semantic representations based on the information encoded in the syntactic representations. properly addressing this issue is beyond the scope of the present paper. what we want to do here is just to point out that one frequently encounters this type of tension both in corpus development and in the use of corpora in linguistic research, and that this is ultimately related to a larger issue in computational linguistics research: what exactly is the relationship between linguistic theory and computational implementation? there is no easy answer to this question, but it seems important to clearly recognize this gap for fruitful development of both theoretical linguistics and computational linguistics in the future. ## conclusion in this paper, we have presented a case study of using the npcmj corpus for theoretical linguistics research. in the first part of the paper, we discussed some methodological issues pertaining to the use of treebanks for theoretical research in order to situate the present case study in a larger context. in the second part of the paper, we have presented a case study of using the npcmj corpus for evaluating theoretical claims about the status of the csc as a syntactic constraint. although the value of treebanks as resource for research is well recognized in the nlp community, treebanks are still novel research tools in the theoretical linguistics community, and their potentials in this latter context don't yet seem to be fully appreciated. we hope that the preliminary discussion of the methodological points and the case study we have presented in this paper will provide a starting point for further examining these methodological issues and for innovative uses of treebanks for approaching theoretical important questions in linguistic research.