Automatic evaluation of syntactic learners in typologically-different languages
Human syntax acquisition involves a system that can learn constraints on possible word sequences in typologically-different human languages. Evaluation of computational syntax acquisition systems typically involves theory-specific or language-specific assumptions that make it difficult to compare results in multiple languages. To address this problem, a bag-of-words incremental generation (BIG) task with an automatic sentence prediction accuracy (SPA) evaluation measure was developed. The BIGSPA task was used to test several learners that incorporated n-gram statistics which are commonly found in statistical approaches to syntax acquisition. In addition, a novel AdjacencyProminence learner, that was based on psycholinguistic work in sentence production and syntax acquisition, was also tested and it was found that this learner yielded the best results in this task on these languages. In general, the BIGSPA task is argued to be a useful platform for comparing explicit theories of syntax acquisition in multiple languages.
syntax acquisition computational linguistics corpora syntax evaluation linguistic typology