Patent ID: 8990069
Filing Date: 2015-03-24
Classification: G06F

Abstract:
1. A computer-implemented method, comprising: receiving, at a server including one or more processors, a phrase table for statistical machine translation, the phrase table including a plurality of phrase pairs corresponding to one or more pairs of languages; determining, at the server, a redundant set of the plurality of phrase pairs, the redundant set of the plurality of phrase pairs including each of the plurality of phrase pairs having a phrase matching a phrase of another of the plurality of phrase pairs, wherein each phrase of the plurality of phrase pairs includes two or more sub-phrases; dividing, at the server, the redundant set of the plurality of phrase pairs into two of more subsets based on a number of sub-phrases in each phrase pair of the redundant set of the plurality of phrase pairs, wherein each subset corresponds to a specific number of sub-phrases; calculating, at the server, a first probability for each specific phrase pair of the redundant set of the plurality of phrase pairs, the first probability indicating a likelihood that its corresponding specific phrase pair is an appropriate translation; calculating, at the server, a second probability for each specific phrase pair based on third probabilities of the sub-phrases of the specific phrase pair, the second and third probabilities each indicating a likelihood that a corresponding specific phrase pair or sub-phrase is an appropriate translation; calculating, at the server, a frequency-weighted difference between a logarithm of the first probability and a logarithm of the second probability to obtain a value for each specific phrase pair; comparing, at the server, the value for each specific phrase pair to a probability threshold indicative of an acceptable change in probability distribution of the phrase table; selecting, at the server, each specific phrase pair when its corresponding value is less than the probability threshold to obtain the one or more selected phrase pairs, wherein the one or more selected phrase pairs for each subset of the redundant set of the plurality of phrase pairs are obtained sequentially from a subset corresponding to a lowest number of sub-phrases to a subset corresponding to a highest number of sub-phrases; and removing, at the server, the one or more selected phrase pairs from the phrase table to obtain a modified phrase table.