In this work we address the challenge of augmenting n-gram language models according to prior linguistic intuitions. We argue that the family of hierarchical Pitman-Yor language models is an attractive vehicle through which to address the problem, and demonstrate the approach by proposing a model for German compounds. In our empirical evaluation the model outperforms a modified Kneser-Ney n-gram model in test set perplexity. When used as part of a translation system, the proposed language model matches the baseline BLEU score for English→German while improving the precision with which compounds are output. We find that an approximate inference technique inspired by the Bayesian interpretation of Kneser-Ney smoothing (Teh, 2006 ) offers a way to drastically reduce model training time with negligible impact on translation quality. TITLE AND ABSTRACT IN AFRIKAANS Hierdie werk neem uitdagings rondom die uitbreiding van n-gramtaalmodelle volgens voorafgaande linguistieke intuïsie onder die loep. Ons voer aan dat die familie van hiërargiese Pitman-Yor taalmodelle 'n wenslike stuk gereedskap is om hierdie probleem mee aan te pak en formuleer 'n model van Duitse saamgestelde woorde om die benadering te demonstreer. Met behulp van 'n empiriese evaluering bevind ons dat die model in terme van toetsdataperpleksiteit beter vaar as die aangepaste Kneser-Ney n-grammodel. As onderdeel van 'n Engels→Duits-vertalingstelsel behaal die model in terme van die BLEU-metriek dieselfde vertaalafvoerkwaliteit as die kontrole stelsel en genereer saamgestelde woorde teen 'n hoër presisie. Verder stel ons vas dat 'n benaderde inferensietegniek, geïnspireer deur die Bayes-interpretasie van Kneser-Ney-gladstryking (Teh, 2006) , gebruik kan word om die modelberamingtyd drasties te verminder sonder wesenlike impak op die vertaalafvoerkwaliteit.
REF discussed the approach of using customized language models to target German compounds.
Bayesian Language Modelling of German Compounds
COLING
Computer Science