The BBN Crosslingual Topic Detection and Tracking System

This was the first year that the TDT program included a required crosslingual test: English and Mandarin. Most of our work, therefore, was to adapt our tracking and detection systems to work on a corpus of documents in these two languages. To this end, we worked both on quick, adequate translation, and the modifications necessary to our systems to attain good performance in this crosslingual domain. We started by building simple term translation systems, and ended with more complicated ones that made sensible use of knowledge of the target language. Working from a parallel corpus of aligned sentences, we estimated prior term translation probabilities. Additionally, we devised an algorithm for iteratively refining a translation, via word co-occurrence statistics, in order to choose the most “consistent” translation. The tracking and detection systems themselves were left largely unchanged, with the exception that we estimated score normalization statistics separately for documents in different languages. We were pleasantly surprised to find that the performance on tracking and detection using our translation system was about the same as when we availed ourselves of the supplied machine translations of Mandarin. 1. Translating for TDT It was our belief that the introduction of cross-lingual tasks into the TDT evaluation meant we needed to build a translation system. It is worth noting that both time and resources were limited. We had only a few months to do this work, and were provided the sort of resources you’d expect when asked to work in a sparse language: an errorful translation dictionary, a parallel corpus not quite from the right domain, various mismatched word lists and concordances. But this seemed to be the main thrust of TDT3: how well and how quickly can you adapt your TDT systems to work on an additional language, given limited resources? 1.1. A Simple Term Translation System We decided that the first pass at a solution should be to translate all the Mandarin documents into English using a bilingual dictionary and then run our monolingual tracking and detection systems as usual. Obviously, this is not going to give the best result. But, since it is the easiest thing to do, it is important to have it as a benchmark against which to compare other, more elaborate solutions. Our translation system used a very crude algorithm. 1. Segment the original Mandarin document into words. 2. Look up each Mandarin word in the bilingual dictionary. If it is in the dictionary, make a bag of all the English words from all the translations. Else, throw it away. 3. Each Mandarin document is just a big bag of English words. We were fortunate to have available two systems for segmenting Mandarin into words, one provided by the LDC and another we at BBN had developed as part of a Mandarin Information Extraction system. Upon inspection, the output of the LDC segmenter appeared more appropriate for input to a translation system. The BBN segmenter tended to leave identities (persons, locations, organization, etc) unsegmented. This is a valuable behaviour when namefinding, but it often renders these crucial terms untranslatable. In addition to trying these different programs for segmenting the Mandarin, we also experimented with segmenting greedily (always take the longest word next that is in the dictionary), and segmenting all-ways at once (all substrings of less than M characters). Notice at least two obvious but significant failings of this sort of simple-minded approach to term translation. 1. We are throwing away too much of the original Mandarin document. Our estimate is that 25% of the words don’t translate. Only about 5% are due to errors in segmentation. About half are names. 2. By translating a Mandarin word as the entirety of the English half of its dictionary entry, we are weighting the words in the translation in precisely the wrong way. Common words typically translate many ways. Uncommon words typically translate very few ways. Together, these two facts conspire to mean that we are increasing the weight for common words, and decreasing it for uncommon words. This is a bad idea. Common words are generally not useful for TDT, while uncommon words are generally very important. The first of these problems can only be addressed by increasing the coverage of the Mandarin words by the term translation system. The second problem is best solved by introducing the notion of translating with probability. 1.2. Extended Word Lookup Translation We can solve the coverage problem by translating and segment jointly, i.e. choosing a segmentation that increases dictionary coverage. We decided to start with the LDC segmentation and then repair it. Roughly half of the untranslated words are names. In many cases, the correct translation for a Chinese name into English is simply the Pinyin spelling for the sequence of characters. We made use of this target language knowledge as well as a list of Chinese surnames, to refine our dictionary lookup algorithm. 1. If the untranslated word begins with a Chinese surname, write out the surname and given name in Pinyin. 2. Else greedily subsegment using the dictionary. This solution does increase coverage; every word in the original now translates. However, we still have the word-weighting problem. 1.3. Probabilistic Term Translation A coarse solution to the word weighting problem is simply to mandate that the English translation have the same total number of words as the original Mandarin. We can achieve this by giving each English word a fractional count proportional to the total number of words in the English translation. This will mean that the three character word that translates only one way (in the bilingual dictionary we had to work with), as “Mao Zedong”, will result in two English words with half a single count each. Likewise, the two character word that translates eight ways as various forms of “haven’t” and “to not be” will result in thirteen English words with 1=13 of a count each. A better solution involves estimating non-uniform prior translation probabilities for these individual words. Not all translations are equally likely. For instance, the Chinese word for “America” translates ten ways. Clearly, though, “America” is a more likely translation than “yankeedom. ” We implemented an iterative procedure to estimate non-uniform prior translation probabilities using the observations of aligned sentences in the parallel corpora of Hong Kong Laws. 1. Initialize P (Ej jC) = 1 Nt(C) 2. Count C(Ej ; C) =PSc;Se;C2Sc;Ej2Se P (Ej jC) Pi P (EijC) 3. Re-estimate P (EjjC) = aP (EjjC) + (1 a) C(Ej ;C) Pi C(Ei;C) using a = x x+Pi C(Ei;C) .