Cross-LingualLexical Triggers in Statistical LanguageModeling

We proposenew methodsto take advantage of text in resource-richlanguages to sharpenstatisticallanguagemodelsin resource-deficient languages.We achieve this throughan extensionof the method of lexical triggers to the cross-language problem,andby developinga likelihoodbasedadaptationschemefor combining a trigger model with an -gram model. We describethe applicationof suchlanguagemodelsfor automaticspeechrecognition. By exploitingaside-corpusof contemporaneousEnglish news articles for adaptinga staticChineselanguagemodel to transcribeMandarinnews stories,we demonstratesignificantreductionsin both perplexity and recognition errors. We alsocompareour cross-lingualadaptation schemeto monolingual languagemodel adaptation, andto analternatemethodfor exploiting cross-lingualcues, via crosslingual informationretrieval andmachine translation,proposedelsewhere. 1 Data Sparsenessin LanguageModeling Statistical techniqueshave been remarkablysuccessfulin automaticspeechrecognition(ASR) and naturallanguageprocessing(NLP) over thelasttwo decades.This success, however, dependscrucially Thisresearchwassupportedby theNationalScienceFoundation(via GrantNo ̄ ITR-0225656andIIS-9982329)andthe Officeof Naval Research(via ContractNo ̄ N00014-01-1-0685). on the availability of accurateand large amounts of suitablyannotatedtrainingdataandit is difficult to build a usablestatisticalmodel in their absence. Most of the success, therefore,hasbeenwitnessed in the so called resour ce-rich languages.More recently, therehasbeenan increasinginterestin languagessuchasMandarinandArabic for ASR and NLP, anddataresourcesarebeingcreatedfor them at considerablecost. The data-resourcebottleneck, however, is likely to remainfor a majority of the world’s languagesin theforeseeablefuture. Methodshave beenproposedto bootstrapacoustic modelsfor ASR in resourcedeficientlanguages by reusingacousticmodelsfrom resource-richlanguages(Schultz and Waibel, 1998; Byrne et al., 2000). Morphologicalanalyzers, noun-phrasechunkers, POStaggers,etc., have also beendeveloped for resourcedeficientlanguagesby exploiting translatedor parallel text (Yarowsky et al., 2001). Khudanpur and Kim (2002) recently proposedusing cross-lingualinformationretrieval (CLIR) andmachinetranslation(MT) to improve a statisticallanguagemodel(LM) in a resource-deficient language by exploiting copiousamountsof text available in resource-richlanguages.Whentranscribinga news story in a resource-deficient language,their core ideais to usethefirst passoutputof a rudimentary ASRsystemasaqueryfor CLIR, identify acontemporaneousEnglishdocumentonthatnewstopic,followedby MT to provide a roughtranslationwhich, evenif not fluent,is adequateto updateestimatesof word frequenciesandtheLM vocabulary. They report up to a 28%reductionin perplexity onChinese text from theHongKongNewscorpus. Report Documentation Page Form Approved OMB No. 0704-0188 Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information, including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it does not display a currently valid OMB control number.