gaof23/citation_sentiment_corpus huggingface dataset:
We have: )|(),|(),|( )|,,()|( 21 21 trictrictric trictritri erpercpercp ecrcpecp = = (6) Assumption 2: For an English triple tri e, assume that i c only depends on {1,2}) (i  i e, and c r only depends on e r . Equation (6) is rewritten as: )|()|()|( )|(),|(),|()|( 2211 21 ec trietrictrictritri rrpecpecp erpercpercpecp = = (7) Notice that )|( 11 ecp and )|( 22 ecp are translation probabilities within triples, they are different from the unrestricted probabilities such as the ones in IBM models (Brown et al. , 1993).
In order to filter some noise caused by the error alignment links, we only retain those translation pairs whose translation probabilities are above a threshold 1 D 1  or co-occurring frequencies are above a threshold 2  . When we train the IBM statistical word alignment model with a limited bilingual corpus in the specific domain, we build another translation dictionary with the same method as for the dictionary . But we adopt a different filtering strategy for the translation dictionary . We use log-likelihood ratio to estimate the association strength of each translation pair because Dunning (1993) proved that log-likelihood ratio performed very well on small-scale data.
= ==              = = m aj j m j aj l i i l i ii m j j mlajdeft en pp m ap 0:1 11 1 2 0 0 0 ),( ),,|()|( ! )|( )|,Pr()|,( 00       eef (3) 1 A cept is defined as the set of target words connected to a source word (Brown et al. , 1993).
2.2 Using Log-Likelihood-Ratios to Estimate Word Translation Probabilities Our method for computing the probabilistic translation lexicon LLR-Lex is based on the the Log2http://www.fjoch.com/GIZA++.html Likelihood-Ratio (LLR) statistic (Dunning, 1993), which has also been used by Moore (2004a; 2004b) and Melamed (2000) as a measure of word association.
64 Table 1: Subjects of "employ" with highest likelihood ratio word freq logA word freq logA bRG 64 50.4 plant 14 31.0 company 27 28.6 operation 8 23.0 industry 9 14.6 firm 8 13.5 pirate 2 12.1 unit 9 9.32 shift 3 8.48 postal service 2 7.73 machine 3 6.56 corporation 3 6.47 manufacturer 3 6.21 insurance company 2 6.06 aerospace 2 5.81 memory device 1 5.79 department 3 5.55 foreign office 1 5.41 enterprise 2 5.39 pilot 2 5.37 *ORG includes all proper names recognized as organizations The logA column are their likelihood ratios (Dunning, 1993).
Not only many combinations are found in the corpus, many of them have very similar mutual information values to that of 318 Table 2: economic impact verb economic financial political social budgetary ecological economic economic economic economic economic economic economic economic economic object impact impact impact impact impact impact effect implication consequence significance fallout repercussion potential ramification risk mutual freq info 171 1.85 127 1.72 46 0.50 15 0.94 8 3.20 4 2.59 84 0.70 17 0.80 59 1.88 10 0.84 7 1.66 7 1.84 27 1.24 8 2.19 17 -0.33 nomial distribution can be accurately approximated by a normal distribution (Dunning, 1993).
The different approaches (e.g. Brent, !991, 1993; Ushioda et al. , 1993; Briscoe and Carroll, 1997; Manning, 1993; Carroll and Rooth, 1998; Gahl, 1998; Lapata, 1999; Sarkar and Zeman, 2000) vary largely according to the methods used and the number of SCFS being extracted.
Method Number of frames Number of verbs Linguistic resources F-Score (evaluation based on a gold standard) Coverage on a corpus C. Manning (1993) 19 200 POS tagger + simple finite state parser 58 T. Briscoe & J. Carroll (1997) 161 14 Full parser 55 A. Sarkar & D. Zeman (2000) 137 914 Annotated treebank 88 D. Kawahara et al.
For each cell in the contingency table, the expected counts are: mi j = ni+n+ jn++ . The measures are calculated as (Pedersen, 1996): 2 = i;j (ni j mi j) 2 mi j LL = 2 i;j log2 n 2i j mi j Log-likelihood ratios (Dunning, 1993) are more appropriate for sparse data than chi-square.
2.2 Corpus occurrence In order to get a feel for the relative frequency of VPCs in the corpus targeted for extraction, namely 0 5 10 15 20 25 30 35 40 0 10 20 30 40 50 60 70 VPC types (%) Corpus frequency Figure 1: Frequency distribution of VPCs in the WSJ Tagger correctextracted Prec Rec Ffl=1 Brill 135135 1.000 0.177 0.301 Penn 667800 0.834 0.565 0.673 Table 1: POS-based extraction results the WSJ section of the Penn Treebank, we took a random sample of 200 VPCs from the Alvey Natural Language Tools grammar (Grover et al. , 1993) and did a manual corpus search for each.
For this present work, we use Dunnings log-likelihood ratio statistics (Dunning, 1993) defined as follows: sim = aloga+blogb+clogc+dlogd (a+b)log(a+b)(a+c)log(a+c) (b+d)log(b+d)(c+d)log(c+d) +(a+b+c+d)log(a+b+c+d) For each bilingual pattern EiJj, we compute its similarity score and qualify it as a bilingual sequence-to-sequence correspondence if no equally strong or stronger association for monolingual constituent is found.
1 minority report 2 box office 3 scooby doo 4 sixth sense 5 national guard 6 bourne identity 7 air national guard 8 united states 9 phantom menace 10 special effects 11 hotel room 12 comic book 13 blair witch project 14 short story 15 real life 16 jude law 17 iron giant 18 bin laden 19 black people 20 opening weekend 21 bad guy 22 country bears 23 mans man 24 long time 25 spoiler space 26 empire strikes back 27 top ten 28 politically correct 29 white people 30 tv show 31 bad guys 32 freddie prinze jr 33 monsters ball 34 good thing 35 evil minions 36 big screen 37 political correctness 38 martial arts 39 supreme court 40 beautiful mind Figure 7: Result of re-ranking output from the phrase extension module 6.4 Revisiting unigram informativeness An alternative approach to calculate informativeness from the foreground LM and the background LM is just to take the ratio of likelihood scores, a11 fga9a54a86 a15 a23 a11 bga9a54a86 a15 . This is a smoothed version of relative frequency ratio which is commonly used to find subject-specific terms (Damerau, 1993).
alpha 0 0.1 0.2 0.3 0.4 0.5 Freq=2 13555 13093 12235 11061 10803 10458 Freq=3 4203 3953 3616 3118 2753 2384 Freq=4 1952 1839 1649 1350 1166 960 Freq=5 1091 1019 917 743 608 511 Freq>2 2869 2699 2488 2070 1666 1307 TOTAL 23670 22603 20905 18342 16996 15620 alpha 0.6 0.7 0.8 0.9 1.0 Freq=2 10011 9631 9596 9554 9031 Freq=3 2088 1858 1730 1685 1678 Freq=4 766 617 524 485 468 Freq=5 392 276 232 202 189 Freq>2 1000 796 627 517 439 TOTAL 14257 13178 12709 12443 11805 Table 7: Number of extracted MWUs by frequency 6.2 Qualitative Analysis As many authors assess (Frank Smadja, 1993; John Justeson and Slava Katz, 1995), deciding whether a sequence of words is a multiword unit or not is a tricky problem.
Candidate term Segment result of GPWS for one sentence, in which term appears   / / / / /  /   / / / / / /  / /   / / / / / / /  / / / / / /   / / / / / / Table 2: Examples of candidates eliminated by GPWS 5 Relative frequency ratio against background corpus Relative frequency ratio (RFR) is a useful method to be used to discover characteristic linguistic phenomena of a corpus when compared with another (Damerau, 1993).
We use the log likelihood ratio (LLR) (Dunning 1993) given by -2log 2 (H o (p;k 1,n 1,k 2,n 2 )/H a (p 1,p 2 ;n 1,k 1,n 2,k 2 )) LLR measures the extent to which a hypothesized model of the distribution of cell counts, H a, differs from the null hypothesis, H o (namely, that the percentage of documents containing this term is the same in both corpora).
2 Related Work The issue of MWE processing has attracted much attention from the Natural Language Processing (NLP) community, including Smadja, 1993; Dagan and Church, 1994; Daille, 1995; 1995; McEnery et al. , 1997; Wu, 1997; Michiels and Dufour, 1998; Maynard and Ananiadou, 2000; Merkel and Andersson, 2000; Piao and McEnery, 2001; Sag et al. , 2001; Tanaka and Baldwin, 2003; Dias, 2003; Baldwin et al. , 2003; Nivre and Nilsson, 2004 Pereira et al,.
This statistic is given by -2 log A = 2(log L(p1, kl, hi) log L(p2, k2, n2)-log L(p, kl, R1)--log L(p, k2, n2)), where log LCo, k, n) = k logp + (n k)log(1 -p), and Pl = ~, P2 = ~, P =,~',~; (For a detailed description of the statistic used, see (Dunning, 1993)).
3.1 The gender/animaticity statistics After we have identified the correct antecedents it is a simple counting procedure to compute P(p\[wa) where wa is in the correct antecedent for the pronoun p (Note the pronouns are grouped by their gender): \[ wain the antecedent for p \[ P(pl o) = When there are multiple relevant words in the antecedent we apply the likelihood test designed by Dunning (1993) on all the words in the candidate NP.
C c C, p(C\]v,r) is just the probability of the disjunction of the concepts in C; that is, = Zp(clv, r) cEC In order to see how p(clv,r) relates to the input data, note that given a concept c, verb v and argument position r, a noun can be generated according to the distribution p(n\[c, v, r), where p(nlc, v, r) = 1 nEsyn(c) Now we have a model for the input data: p(n, v, r) = p(v,r)p(niv,r) = p(v,r) p(clv, rlp(ntc, v,r) cecn(n) Note that for c  cn(n), p(nlc, v, r) = O. The association norm (and similar measures such as the mutual information score) have been criticised (Dunning, 1993) because these scores can be greatly over-estimated when frequency counts are low.
have been used in statistical machine translation (Brown et al. , 1990), terminology research and translation aids (Isabelle, 1992; Ogden and Gonzales, 1993; van der Eijk, 1993), bilingual lexicography (Klavans and Tzoukermann, 1990; Smadja, 1992), word-sense disambiguation (Brown et al. , 1991b; Gale et al. , 1992) and information retrieval in a multilingual environment (Landauer and Littman, 1990).
Iegar(ling l;his l;ypu of (:olloeation, the approaches till ilOW could be divi(led inl;o t;wo groups: those thai; do uo(, refer to s'ttbstrings of colloco, l, ions as a l)arti(:ular problem, (Church and lla.nks, t99(); Kim and Cho, 1993; Nagao and Mori, 1994), and those t.hat; do (Kita et al. , t994; Smadja, 1993; lkchara et al. , 1995; Kjelhner, 11994).
Since Odds = P/(1  P), we multiply both sides of Definition 3 by (1P(U|E))1 to obtain, P(U|E) 1P(U|E) = P(E|U)P(U) P(E)(1P(U|E)) (7) By substituting Equation 6 in Equation 7 and later, applying the multiplication rule P(U|E)P(E) = P(E|U)P(U) to it, we will obtain: P(U|E) P(U|E) = P(E|U)P(U) P(E|U)P(U) (8) We proceed to take the log of the odds in Equation 8 (i.e. logit) to get: log P(E|U)P(E|U) = log P(U|E)P(U|E) log P(U)P(U) (9) While it is obvious that certain words tend to cooccur more frequently than others (i.e. idioms and collocations), such phenomena are largely arbitrary (Smadja, 1993).
There has been a growing interest in corpus-based approaches which retrieve collocations from large corpora (Nagao and Mori, 1994), (Ikehara et al. , 1996) (Kupiec, 1993), (Fung, 1995), (Kitamura and Matsumoto, 1996), (Smadja, 1993), (Smadja et al. , 1996), (Haruno et al. , 1996).
Co-occurrence information between neighboring words and words in the same sentence has been used in phrase extraction (Smadja, 1993; Fung and Wu, 1994), phrasal translation (Smadja et al. , 1996; Kupiec, 1993; Wu, 1995; Dagan and Church, 1994), target word selection (Liu and Li, 1997; Tanaka and Iwasaki, 1996), domain word translation (Fung and Lo, 1998; Fung, 1998), sense disambiguation (Brown et al. , 1991; Dagan et al. , 1991; Dagan and Itai, 1994; Gale et al. , 1992a; Gale et al. , 1992b; Gale et al. , 1992c; Shiitze, 1992; Gale et al. , 1993; Yarowsky, 1995), and even recently for query translation in cross-language IR as well (Ballesteros and Croft, 1998).
Co-occurrence statistics is collected from either bilingual parallel and 334 non-parallel corpora (Smadja et al. , 1996; Kupiec, 1993; Wu, 1995; Tanaka and Iwasaki, 1996; Fung and Lo, 1998), or monolingual corpora (Smadja, 1993; Fung and Wu, 1994; Liu and Li, 1997; Shiitze, 1992; Yarowsky, 1995).
Related Works Generally speaking, approaches to MWE extraction proposed so far can be divided into three categories: a) statistical approaches based on frequency and co-occurrence affinity, b) knowledgebased or symbolic approaches using parsers, lexicons and language filters, and c) hybrid approaches combining different methods (Smadja 1993; Dagan and Church 1994; Daille 1995; McEnery et al. 1997; Wu 1997; Wermter et al. 1997; Michiels and Dufour 1998; Merkel and Andersson 2000; Piao and McEnery 2001; Sag et al. 2001a, 2001b; Biber et al. 2003).
Morphologicaltoolssuch as lemmatizers andPOStaggersarebeingcommonlyusedin extractionsystems;they areemployedbothfordealingwithtext variationandfor validatingthe candidatepairs: combinationsof functionwordsare typicallyruledout (Justesonand Katz, 1995),as are the ungrammaticalcombinationsin the systemsthatmake useofparsers(ChurchandHanks, 1990;Smadja,1993;Basilietal.
Given the motivations for performing a linguistically-informedextraction whichwere also put forth, among others, by Church and Hanks(1990,25), Smadja(1993,151) and Heid (1994)  and given the recent developmentof linguisticanalysistools,itseemsplausiblethatthe linguisticstructurewill be more and more taken intoaccountbycollocationextractionsystems.
3 OverviewofExtractionWork 3.1 English As one mightexpect,the bulk of the collocation extractionwork concernsthe English language: (Choueka,1988;Churchet al. ,1989;Churchand Hanks,1990; Smadja,1993; Justesonand Katz, 1995;Kjellmer, 1994;Sinclair, 1995;Lin,1998), amongmany others1.
Smadja(1993)employsthez-scoreinconjunction with several heuristics(e.g. , the systematic occurrenceof two lexical items at the same distanceintext)andextractspredicativecollocations, 1E.g.,(Frantziet al. ,2000;Pearce,2001;Goldmanet al. , 2001;ZaiuInkpenandHirst,2002;Dias,2003;Seretanetal.
2.1 The Evaluator The evaluator is a function p(t\[t', s) which assigns to each target-text unit t an estimate of its probability given a source text s and the tokens t' which precede t in the current translation of s. 1 Our approach to modeling this distribution is based to a large extent on that of the IBM group (Brown et al. , 1993), but it differs in one significant aspect: whereas the IBM model involves a "noisy channel" decomposition, we use a linear combination of separate predictions from a language model p(tlt ~) and a translation model p(tls ).
3 Bilingual Task: An Application for Word Alignment 3.1 Sentence and word alignment Bilingual alignment methods (Warwick et al. , 1990; Brown et al. , 1991a; Brown et al. , 1993; Gale and Church, 1991b; Gale and Church, 1991a; Kay and Roscheisen, 1993; Simard et al. , 1992; Church, 1993; Kupiec, 1993a; Matsumoto et al. , 1993; Dagan et al. , 1993).
Thus, a lot of alignment techniques have been suggested at; the sentence (Gale et al. , 1993), phrase (Shin et al. , 1996), nomt t)hrase (Kupiec, 1993), word (Brown et al. , 1993; Berger et al. , 1996; Melamed, 1997), collocation (Smadja et al. , 1996) and terminology level.
3 Model 1 and Model 2 l~cl)lacing the (l(~,t)endence on aj-l in the HMM alignment mo(M I)y a del)endence on j, we olltain a model wlfich (:an lie seen as a zero-order Hid(l(mMarkov Model which is similar to Model 2 1)rot)ose(t t/y (Brown et al. , 1993).
The assumptions we made were the following:  a lexical token in one half of the translation unit (TU) corresponds to at most one non-empty lexical unit in the other half of the TU; this is the 1:1 mapping assumption which underlines the work of many other researchers (Ahrenberg et al (2000), Brew and McKelvie (1996), Hiemstra (1996), Kay and Rscheisen (1993), Tiedmann (1998), Melamed (2001) etc);  a polysemous lexical token, if used several times in the same TU, is used with the same meaning; this assumption is explicitly used by Gale and Church (1991), Melamed (2001) and implicitly by all the previously mentioned authors;  a lexical token in one part of a TU can be aligned to a lexical token in the other part of the TU only if the two tokens have compatible types (part-of-speech); in most cases, compatibility reduces to the same POS, but it is also possible to define other compatibility mappings (e.g. participles or gerunds in English are quite often translated as adjectives or nouns in Romanian and vice-versa);  although the word order is not an invariant of translation, it is not random either (Ahrenberg et al (2000)); when two or more candidate translation pairs are equally scored, the one containing tokens which are closer in relative position are preferred.
Besides being used in SMT, it is also used in translation lexicon building (Melamed 1996), transfer rule learning (Menezes and Richardson 2001), example-based machine translation (Somers 1999), etc. In previous alignment methods, some researches modeled the alignments as hidden parameters in a statistical translation model (Brown et al. 1993; Och and Ney 2000) or directly modeled them given the sentence pairs (Cherry and Lin 2003).
By introducing the hidden word alignment variable a  (Brown et al., 1993), the optimal translation can be searched for based on the following criterion: * 1 , arg max( ( , , )) M mm m ea eh = = efa             (1) where  is a string of phrases in the target language, e f  fa    is the source language string of phrases,  he  are feature functions, weights (, , ) m m  are typically optimized to maximize the scoring function (Och, 2003).
Given a manually compiled lexicon containing words and their relative frequencies Ps(fprimej), the best segmentationfJ1 is the one that maximizes the joint probability of all words in the sentence, with the assumption that words are independent of each other1: fJ1 = argmax fprimeJprime1 Pr(fprimeJprime1 |cK1 )  argmax fprimeJprime1 Jprimeproductdisplay j=1 Ps(fprimej), where the maximization is taken over Chinese word sequences whose character sequence is cK1 . 2.2 Translation system Once we have segmented the Chinese sentences into words, we train standard alignment models in both directions with GIZA++ (Och and Ney, 2002) using models of IBM-1 (Brown et al., 1993), HMM (Vogel et al., 1996) and IBM-4 (Brown et al., 1993).
Based on these grammars, a great number of SMT models have been recently proposed, including string-to-string model (Synchronous FSG) (Brown et al., 1993; Koehn et al., 2003), tree-to-string model (TSG-string) (Huang et al., 2006; Liu et al., 2006; Liu et al., 2007), string-totree model (string-CFG/TSG) (Yamada and Knight, 2001; Galley et al., 2006; Marcu et al., 2006), tree-to-tree model (Synchronous CFG/TSG, Data-Oriented Translation) (Chiang, 2005; Cowan et al., 2006; Eisner, 2003; Ding and Palmer, 2005; Zhang et al., 2007; Bod, 2007; Quirk wt al., 2005; Poutsma, 2000; Hearne and Way, 2003) and so on.
So fitr, we have implemented the following,: sentence ~dignment btLsed-on word correspondence information, word correspondence estimation by cooccnl'rence-ffequency-based methods in GMe mid Church (19.~H) and Kay and R6scheisen (1993), structured Imttehlng of parallel sentences (Matsumoto et a l. , 1993), and case Dame acquisition of Japanese verbs (Utsuro et al. , 1993).
Motivation There have been quite a number of recent papers on parallel text: Brown et al (1990, 1991, 1993), Chen (1993), Church (1993), Church et al (1993), Dagan et al (1993), Gale and Church (1991, 1993), Isabelle (1992), Kay and Rgsenschein (1993), Klavans and Tzoukermann (1990), Kupiec (1993), Matsumoto (1991), Ogden and Gonzales (1993), Shemtov (1993), Simard et al (1992), WarwickArmstrong and Russell (1990), Wu (to appear).
, 1993; Graham et al. , 1980) where K is the number of distinct nonternfinal symbols in the gramma.r G. We ca.n expect a. very etfide.nt pa.rser tbr our pa.tterns, r The input string ca.n a.lso be scanned to reduce the number of relewmt gramma.r rules before pa.rsing, e The combined process is a.lso known as offlineparsing in LTAC,.
3.1 The traditional IBM alignment model IBM Model 4 (Brown et al. , 1993) learns a set of 4 probability tables to compute p(f|e) given a foreign sentence f and its target translation e via the following (greatly simplified) generative story: 361 NP-C NPB NPB NNP taiwan POS s NN surplus PP IN in NP-C NPB NN trade PP IN between NP-C NPB DT the CD two NNS shores FTD0 GR G4E7 DYBG EL DIDV TAIWAN IN TWO-SHORES TRADE MIDDLE SURPLUS R1: NP-C NPB x0:NPB x1:NN x2:PP x0 x2EL x1 R10: NP-C NPB x0:NPB x1:NN x2:PP x0 x2 x1 R10: NP-C NPB x0:NPB x1:NN x2:PP x0 x2 x1 R2: NPB NNP taiwan POS s FTD0 R11: NPB x0:NNP POS s x0 R17: NPB NNP taiwan x0:POS x0 R12: NNP taiwan FTD0 R18: POS s FTD0 R3: PP x0:IN x1:NP-C x0 x1 R13: PP IN in x0:NP-C GR x0EL R19: PP IN in x0:NP-C x0 R4: IN in  GR R5: NP-C x0:NPB x1:PP  x1 x0 R5: NP-C x0:NPB x1:PP x1 x0 R20: NP-C x0:NPB PP x1:IN x2:NP-C x2 x0 x1 R6: PP IN between NP-C NPB DT the CD two NNS shores G4E7 R14: PP IN between x0:NP-C x0 R21: IN between EL R15: NP-C x0:NPB x0 R15: NP-C x0:NPB x0 R16: NPB DT the CD two NNS shores G4E7 R22: NPB x0:DT CD two x1:NNS x0 x1 R23: NNS shores G4E7 R24: DT the GR R7: NPB x0:NN x0 R7: NPB x0:NN x0 R7: NPB x0:NN x0 R8: NN trade DYBG R9: NN surplus DIDV R8: NN trade DYBG R9: NN surplus DIDV R8: NN trade DYBG R9: NN surplus DIDV Figure 2: A (English tree, Chinese string) pair and three different sets of multilevel tree-to-string rules that can explain it; the first set is obtained from bootstrap alignments, the second from this papers re-alignment procedure, and the third is a viable, if poor quality, alternative that is not learned.
1 Introduction Given a source-language (e.g. , French) sentence f, the problem of machine translation is to automatically produce a target-language (e.g. , English) translation e. The mathematics of the problem were formalized by (Brown et al. , 1993), and re-formulated by (Och and Ney, 2004) in terms of the optimization e = arg maxe Msummationdisplay m=1 mhm(e,f) (1) where fhm(e,f)g is a set of M feature functions and fmg a set of weights.
The words with the highest association probabilities are chosen as acquired words for entity e. 4.1 Base Model I Using the translation model I (Brown et al., 1993), where each word is equally likely to be aligned with each entity, we have p(w|e) = 1(l + 1)m mproductdisplay j=1 lsummationdisplay i=0 p(wj|ei) (1) where l and m are the lengths of entity and word sequences respectively.
4.2 Base Model II Using the translation model II (Brown et al., 1993), where alignments are dependent on word/entity positions and word/entity sequence lengths, we have p(w|e) = mproductdisplay j=1 lsummationdisplay i=0 p(aj = i|j,m,l)p(wj|ei) (2) where aj = i means that wj is aligned with ei.
By introducing the hidden word alignment variable a, the following approximate optimization criterion can be applied for that purpose: e = argmaxe Pr(e | f) = argmaxe summationdisplay a Pr(e,a | f)  argmaxe,a Pr(e,a | f) Exploiting the maximum entropy (Berger et al. , 1996) framework, the conditional distribution Pr(e,a | f) can be determined through suitable real valued functions (called features) hr(e,f,a),r = 1R, and takes the parametric form: p(e,a | f)  exp Rsummationdisplay r=1 rhr(e,f,a)} The ITC-irst system (Chen et al. , 2005) is based on a log-linear model which extends the original IBM Model 4 (Brown et al. , 1993) to phrases (Koehn et al. , 2003; Federico and Bertoldi, 2005).
287 System Train +base Test +base 1 Baseline 87.89 87.89 2 Contrastive 88.70 0.82 88.45 0.56 (5 trials/fold) 3 Contrastive 88.82 0.93 88.55 0.66 (greedy selection) Table 1: Average F1 of 7-way cross-validation To generate the alignments, we used Model 4 (Brown et al., 1993), as implemented in GIZA++ (Och and Ney, 2003).
According to this model, when translating a stringf in the source language into the target language, a string e is chosen out of all target language strings e if it has the maximal probability given f (Brown et al., 1993): e = arg maxe {Pr(e|f)} = arg maxe {Pr(f|e)Pr(e)} where Pr(f|e) is the translation model and Pr(e) is the target language model.
To model p(fJle~;8,.T) we assume the existence of an alignment a J. We assume that every word fj is produced by the word e~j at position aj in the training corpus with the probability P(f~le,~i): J p(f lc ') = 1\] p(L Icon) j=l (7) The word alignment a J is trained automatically using statistical translation models as described in (Brown et al. , 1993; Vogel et al. , 1996).