PCFGs with latent annotations have been shown to be a very effective model for phrase structure parsing . We present a Bayesian model and algorithms based on a Gibbs sampler for parsing with a grammar with latent annotations . For PCFG-LA , we present an additional Gibbs sampler algorithm to learn annotations from training data , which are parse trees with coarse ( unannotated ) symbols . We show that a Gibbs sampling technique is capable of parsing sentences in a wide variety of languages and producing results that are on-par with or surpass previous approaches . Our results for Kinyarwanda and Malagasy in particular demonstrate that low-resource language parsing can benefit substantially from a Bayesian approach . Despite great progress over the past two decades on parsing , relatively little work has considered the problem of creating accurate parsers for low-resource languages . Existing work in this area focuses primarily on approaches that use some form of cross-lingual bootstrapping to improve performance . For instance , Hwa et al . ( 2005 ) use a parallel Chinese / English corpus and an English dependency grammar to induce an annotated Chinese corpus in order to train a Chinese dependency grammar . Kuhn ( 2004b ) also considers the benefits of using multiple languages to induce a monolingual grammar , making use of a measure for data reliability in order to weight training data based on confidence of annotation . Bootstrapping approaches such as these achieve markedly improved results , but they are dependent on the existence of a parallel bilingual corpus . Very few such corpora are readily available , particularly for low-resource languages , and creating such corpora obviously presents a challenge for many practical applications . Kuhn ( 2004a ) shows some of the difficulty in handling low-resource languages by examining various tasks using Q'anjob'al as an example . Another approach is that of Bender et al . ( 2002 ) , who take a more linguistically-motivated approach by making use of linguistic universals to seed newly developed grammars . This substantially reduces the effort by making it unnecessary to learn the basic parameters of a language , but it lacks the robustness of grammars learned from data . Recent work on Probabilistic Context-Free Grammars with latent annotations ( PCFG-LA ) ( Matsuzaki et al . , 2005 ; Petrov et al . , 2006 ) have shown them to be effective models for syntactic parsing , especially when less training material is available ( Liang et al . , 2009 ; Shindo et al . , 2012 ) . The coarse nonterminal symbols found in vanilla PCFGs are refined by latent variables ; these latent annotations can model subtypes of grammar symbols that result in better grammars and enable better estimates of grammar productions . In this paper , we provide a Gibbs sampler for learning PCFG-LA models and show its effectiveness for parsing lowresource languages such as Malagasy and Kinyawanda . Previous PCFG-LA work focuses on the problem of parameter estimation , including expectationmaximization ( EM ) ( Matsuzaki et al . , 2005 ; Petrov et al . , 2006 ) , spectral learning ( Cohen et al . , 2012 ; Cohen et al . , 2013 ) , and variational inference ( Liang et al . , 2009 ; Wang and Blunsom , 2013 ) . Regardless of inference method , previous work has used the same method to parse new sentences : a Viterbi parse under a new sentence-specific PCFG obtained from an approximation of the original grammar ( Matsuzaki et al . , 2005 ) . Here , we provide an alternative approach to parsing new sentences : an extension of the Gibbs sampling algorithm of Johnson et al . ( 2007 ) , which learns rule probabilities in an unsupervised PCFG . We use a Gibbs sampler to collect sampled trees theoretically distributed from the true posterior distribution in order to parse . Priors in a Bayesian model can control the sparsity of grammars ( which the insideoutside algorithm fails to do ) , while naturally incorporating smoothing into the model ( Johnson et al . , 2007 ; Liang et al . , 2009 ) . We also build a Bayesian model for parsing with a treebank , and incorporate information from training data as a prior . Moreover , we extend the Gibbs sampler to learn and parse PCFGs with latent annotations . Learning the latent annotations is a compute-intensive process . We show how a small amount of training data can be used to bootstrap : after running a large number of sampling iterations on a small set , the resulting parameters are used to seed a smaller number of iterations on the full training data . This allows us to employ more latent annotations while maintaining reasonable training times and still making full use of the available training data . To determine the cross-linguistic applicability of these methods , we evaluate on a wide variety of languages with varying amounts of available training data . We use English and Chinese as examples of languages with high data availability , while Italian , Malagasy , and Kinyarwanda provide examples of languages with little available data . We find that our technique comes near state of the art results on large datasets , such as those for Chinese and English , and it provides excellent results on limited datasets -both artificially limited in the case of English , and naturally limited in the case of Italian , Malagasy , and Kinyarwanda . This , combined with its ability to run off-the-shelf on new languages without any supporting materials such as parallel corpora , make it a valuable technique for the parsing of low-resource languages . Our experiments demonstrate that sampling vanilla PCFGs , as well as PCFGs with latent annotations , is feasible with the use of a Gibbs sampler technique and produces results that are in line with previous parsers on controlled test sets . Our results also show that our methods are effective on a wide variety of languages-including two low-resource languageswith no language-specific model modifications needed . Additionally , although not a uniform winner , the Gibbs-PCFG shows a propensity for performing well on naturally small corpora ( here , KIN / MLG ) . The exact reason for this remains slightly unclear , but the fact that a similar advantage is not found for extremely small versions of large corpora indicates that our approach may be particularly well-suited for application in real low-resource environments as opposed to a sim- Having established this procedure and its relative tolerance for low amounts of data , we would like to extend the model to make use of partial bracketing information instead of complete trees , perhaps in the form of Fragmentary Unlabeled Dependency Grammar annotations ( Schneider et al . , 2013 ) . This would allow the sampling procedure to potentially operate using corpora with lighter annotations than full trees , making initial annotation effort not quite as heavy and potentially increasing the amount of available data for low-resource languages . Additionally , using the expert partial annotations to help restrict the sample space could provide good gains in terms of training time .

Challenge: Existing parsing methods for low-resource languages either use bootstrapping which requires parallel corpus that is not often available or a linguistically-motivated approach which lacks robustness.
Approach: They present a Gibbs sampler-based Bayesian approach with a grammar with latent annotations and its extension which learns rules from probabilities in an unsupervised PCFG.
Outcome: The proposed technique performs on par or surpasses previous approaches in many languages but especially performs well in Kinyarwanda and Malagasy with small corpora.