gaof23/citation_sentiment_corpus huggingface dataset:
Table 3: Example compressions Compression AvgLen Rating Baseline 9.70 1.93 BT-2-Step 22.06 3.21 Spade 19.09 3.10 Humans 20.07 3.83 Table 4: Mean ratings for automatic compressions nally, we added a simple baseline compression algorithm proposed by Jing and McKeown (2000) which removed all prepositional phrases, clauses, toinfinitives, and gerunds.
Automatic text summarization approaches have offered reasonably well-performing approximations for identifiying important sentences (Lin and Hovy, 2002; Schiffman et al., 2002; Erkan and Radev, 2004; Mihalcea and Tarau, 2004; Daume III and Marcu, 2006) but, not surprisingly, text (re)generation has been a major challange despite some work on sub-sentential modification (Jing and McKeown, 2000; Knight and Marcu, 2000; Barzilay and McKeown, 2005).
In addition to sentence fusion, compression algorithms (Chandrasekar, Doran, and Bangalore 1996; Grefenstette 1998; Mani, Gates, and Bloedorn 1999; Knight and Marcu 2002; Jing and McKeown 2000; Reizler et al. 2003) and methods for expansion of a multiparallel corpus (Pang, Knight, and Marcu 2003) are other instances of such methods.
Recent comparisons of approaches that can be trained on corpora (van Halteren et al. , 1998; Volk and Schneider, 1998) have shown that in most cases statistical aproaches (Cutting et al. , 1992; Schmid, 1995; Ratnaparkhi, 1996) yield better results than finite-state, rule-based, or memory-based taggers (Brill, 1993; Daelemans et al. , 1996).
4.1 Complete ambiguity classes Ambiguity classes capture the relevant property we are interested in: words with the same category possibilities are grouped together.4 And ambiguity classes have been shown to be successfully employed, in a variety of ways, to improve POS tagging (e.g., Cutting et al., 1992; Daelemans et al., 1996; Dickinson, 2007; Goldberg et al., 2008; Tseng et al., 2005).
157 ena or the linguist's abstraction capabilities (e.g. knowledge about what is relevant in the context), they tend to reach a 95-97% accuracy in the analysis of several languages, in particular English (Marshall 1983; Black et aL 1992; Church 1988; Cutting et al. 1992; de Marcken 1990; DeRose 1988; Hindle 1989; Merialdo 1994; Weischedel et al. 1993; Brill 1992; Samuelsson 1994; Eineborg and Gamb~ick 1994, etc.).
On the other hand, according to the data-driven approach, a frequency-based language model is acquired from corpora and has the forms of ngrams (Church, 1988; Cutting et al. , 1992), rules (Hindle, 1989; Brill, 1995), decision trees (Cardie, 1994; Daelemans et al. , 1996) or neural networks (Schmid, 1994).
Two main approaches have generally been considered: rule-based (Klein and Simmons 1963; Brodda 1982; Paulussen and Martin 1992; Brill et al. 1990) probabilistic (Bahl and Mercer 1976; Debili 1977; Stolz, Tannenbaum, and Carstensen 1965; Marshall 1983; Leech, Garside, and Atwell 1983; Derouault and Merialdo 1986; DeRose 1988; Church 1989; Beale 1988; Marcken 1990; Merialdo 1991; Cutting et al. 1992).
Stochastic taggers use both contextual and morphological information, and the model parameters are usually defined or updated automatically from tagged texts (Cerf-Danon and E1-Beze 1991; Church 1988; Cutting et al. 1992; Dermatas and Kokkinakis 1988, 1990, 1993, 1994; Garside, Leech, and Sampson 1987; Kupiec 1992; Maltese * Department of Electrical Engineering, Wire Communications Laboratory (WCL), University of Patras, 265 00 Patras, Greece.
Part-of-speech tagging is an active area of research; a great deal of work has been done in this area over the past few years (e.g. , Jelinek 1985; Church 1988; Derose 1988; Hindle 1989; DeMarcken 1990; Merialdo 1994; Brill 1992; Black et al. 1992; Cutting et al. 1992; Kupiec 1992; Charniak et al. 1993; Weischedel et al. 1993; Schutze and Singer 1994).
Almost all recent work in developing automatically trained part-of-speech taggers has been on further exploring Markovmodel based tagging (Jelinek 1985; Church 1988; Derose 1988; DeMarcken 1990; Merialdo 1994; Cutting et al. 1992; Kupiec 1992; Charniak et al. 1993; Weischedel et al. 1993; Schutze and Singer 1994).
For English there are many POS taggers, employing machine learning techniques like transformation-based error-driven learning (Brill, 1995), decision trees (Black et al. , 1992), markov model (Cutting et al. 1992), maximum entropy methods (Ratnaparkhi, 1996) etc. There are also taggers which are hybrid using both stochastic and rule-based approaches, such as CLAWS (Garside and Smith, 1997).
There has been a large number of studies in tagging and morphological disambiguation using various techniques such as statistical techniques, e.g., (Church, 1988; Cutting et al. , 1992; DeRose, 1988), constraint-based techniques (Karlsson et al. , 1995; Voutilainen, 1995b; Voutilainen, Heikkil/i, and Anttila, 1992; Voutilainen and Tapanainen, 1993; Oflazer and KuruSz, 1994; Oflazer and Till 1996) and transformation-based techniques (Brilt, 1992; Brill, 1994; Brill, 1995).
Many approaches for POS tagging have been developed in the past, including rule-based tagging (Brill, 1995), HMM taggers (Brants, 2000; Cutting and others, 1992), maximum-entropy models (Rathnaparki, 1996), cyclic dependency networks (Toutanova et al. , 2003), memory-based learning (Daelemans et al. , 1996), etc. All of these approaches require either a large amount of annotated training data (for supervised tagging) or a lexicon listing all possible tags for each word (for unsupervised tagging).
Almost all of the work in the area of automatically trained taggers has explored Markov-model based part of speech tagging \[Jelinek, 1985; Church, 1988; Derose, 1988; DeMarcken, 1990; Cutting et al. , 1992; Kupiec, 1992; Charniak et al. , 1993; Weischedel et al. , 1993; Schutze and Singer, 1994; Lin et al. , 1994; Elworthy, 1994; Merialdo, 1995\].
1 Introduction In the part-of-speech hterature, whether taggers are based on a rule-based approach (Klein and Simmons, 1963), (Brill, 1992), (Voutilainen, 1993), or on a statistical one (Bahl and Mercer, 1976), (Leech et al. , 1983), (Merialdo, 1994), (DeRose, 1988), (Church, 1989), (Cutting et al. , 1992), there is a debate as to whether more attention should be paid to lexical probabilities rather than contextual ones.
Generalized Forward Backward Reestimation Generalization of the Forward and Viterbi Algorithm In English part of speech taggers, the maximization of Equation (1) to get the most likely tag sequence, is accomplished by the Viterbi algorithm (Church, 1988), and the maximum likelihood estimates of the parameters of Equation (2) are obtained from untagged corpus by the ForwardBackward algorithm (Cutting et al. , 1992).
(Cutting et al. , 1992; Feldweg, 1995)), the tagger for grammatical functions works with lexical (1) Selbst besucht ADV VVPP himself visited hat Peter Sabine VAFIN NE NE has Peter Sabine 'Peter never visited Sabine himself' l hie ADV never Figure 2: Example sentence and contextual probability measures PO.(') depending on the category of a mother node (Q).
At the present time, given the key role of window size in determining the selection and apparent strength of associations under the conventional co-occurrence model highlighted here and in the works of Church et al (1991), Rapp (2002), Wang (2005), and Schulte im Walde & Melinger (2008) we would urge that this is an issue which window-driven studies continue to conscientiously address; at the very least, scale is a parameter which findings dependent on distributional phenomena must be qualified in light of.
Eigenvector centrality in particular has been successfully applied to many different types of networks, including hyperlinked web pages (Brin and Page, 1998; Kleinberg, 1998), lexical networks (Erkan and Radev, 2004; Mihalcea and Tarau, 2004; Kurland and Lee, 2005; Kurland and Lee, 2006), and semantic networks (Mihalcea et al. , 2004).
Previous approaches include supervised learning (Hirao et al. , 2002), (Teufel and Moens, 1997), vectorial similarity computed between an initial abstract and sentences in the given document, intradocument similarities (Salton et al. , 1997), or graph algorithms (Mihalcea and Tarau, 2004), (Erkan and Radev, 2004), (Wolf and Gibson, 2004).
Ranking algorithms, such as Kleinbergs HITS algorithm (Kleinberg, 1999) or Googles PageRank (Brin and Page, 1998), have been traditionally and successfully used in Web-link analysis (Brin and Page, 1998), social networks, and more recently in text processing applications (Mihalcea and Tarau, 2004), (Mihalcea et al. , 2004), (Erkan and Radev, 2004).
Inspired by the idea of graph based algorithms to collectively rank and select the best candidate, research efforts in the natural language community have applied graph-based approaches on keyword selection (Mihalcea and Tarau, 2004), text summarization (Erkan and Radev, 2004; Mihalcea, 2004), word sense disambiguation (Mihalcea et al. , 2004; Mihalcea, 2005), sentiment analysis (Pang and Lee, 2004), and sentence retrieval for question answering (Otterbacher et al. , 2005).
Although various approaches to SMT system combination have been explored, including enhanced combination model structure (Rosti et al., 2007), better word alignment between translations (Ayan et al., 2008; He et al., 2008) and improved confusion network construction (Rosti et al., 2008), most previous work simply used the ensemble of SMT systems based on different models and paradigms at hand and did not tackle the issue of how to obtain the ensemble in a principled way.
4 Training This section discusses how to extract our translation rules given a triple nullnull,null null ,nullnull . As we know, the traditional tree-to-string rules can be easily extracted from nullnull,null null ,nullnull  using the algorithm of Mi and Huang (2008) 2 . We would like  2  Mi and Huang (2008) extend the tree-based rule extraction algorithm (Galley et al., 2004) to forest-based by introducing non-deterministic mechanism.
A tree sequence to string rule  174 A tree-sequence to string translation rule in a forest is a triple <L, R, A>, where L is the tree sequence in source language, R is the string containing words and variables in target language, and A is the alignment between the leaf nodes of L and R. This definition is similar to that of (Liu et al. 2007, Zhang et al. 2008a) except our treesequence is defined in forest.
To address this issue, many syntax-based approaches (Yamada and Knight, 2001; Eisner, 2003; Gildea, 2003; Ding and Palmer, 2005; Quirk et al, 2005; Zhang et al, 2007, 2008a; Bod, 2007; Liu et al, 2006, 2007; Hearne and Way, 2003) tend to integrate more syntactic information to enhance the non-contiguous phrase modeling.
Various approaches to word sense division have been proposed in the literature on WSD, including (1) sense numbers in every-day dictionaries (Lesk 1986; Cowie, Guthrie, and Guthrie 1992), (2) automatic or hand-crafted clusters of dictionary senses (Dolan 1994; Bruce and Wiebe 1995; Luk * Department of Computer Science, National Tsing Hua University, Hsinchu 30043, Taiwan, ROC.
405 PRF 1 proposed .383 .437 .408 multinomial mixture .360 .374 .367 Newman (2004) .318 .353 .334 cosine .603 .114 .192 -skew divergence (Lee, 1999) .730 .155 .255 Lins similarity (Lin, 1998) .691 .096 .169 CBC (Lin and Pantel, 2002) .981 .060 .114 Table 3: Precision, recall, and F-measure.
Given a wordq, its set of featuresFq and feature weightswq(f) for f Fq, a common symmetric similarity measure is Lin similarity (Lin, 1998a): Lin(u,v) = summationtext fFuFv[wu(f)+wv(f)]summationtext fFu wu(f)+ summationtext fFv wv(f) where the weight of each feature is the pointwise mutual information (pmi) between the word and the feature: wq(f) =log[Pr(f|q)Pr(f) ].
Discriminative, context-specific training seems to yield a better set of similar predicates, e.g. the highest-ranked contexts for DSPcooc on the verb join,3 lead 1.42, rejoin 1.39, form 1.34, belong to 1.31, found 1.31, quit 1.29, guide 1.19, induct 1.19, launch (subj) 1.18, work at 1.14 give a better SIMS(join) for Equation (1) than the top similarities returned by (Lin, 1998a): participate 0.164, lead 0.150, return to 0.148, say 0.143, rejoin 0.142, sign 0.142, meet 0.142, include 0.141, leave 0.140, work 0.137 Other features are also weighted intuitively.
We also test an MI model inspired by Erk (2007): MISIM(n,v) = log summationdisplay nSIMS(n) Sim(n,n) Pr(v,n ) Pr(v)Pr(n) We gather similar words using Lin (1998a), mining similar verbs from a comparable-sized parsed corpus, and collecting similar nouns from a broader 10 GB corpus of English text.4 We also use Keller and Lapata (2003)s approach to obtaining web-counts.
This similarity score is computed as a max over a number of component scoring functions, some based on external lexical resources, including:  various string similarity functions, of which most are applied to word lemmas  measures of synonymy, hypernymy, antonymy, and semantic relatedness, including a widelyused measure due to Jiang and Conrath (1997), based on manually constructed lexical resources such as WordNet and NomBank  a function based on the well-known distributional similarity metric of Lin (1998), which automatically infers similarity of words and phrases from their distributions in a very large corpus of English text The ability to leverage external lexical resources both manually and automatically constructedis critical to the success of MANLI.
3http://www.openoffice.org Another corpora based method due to Turney and Littman (2003) tries to measure the semantic orientation O(t) for a term t by O(t) = summationdisplay tiS+ PMI(t,ti) summationdisplay tjS PMI(t,tj) where S+ and S are minimal sets of polar terms that contain prototypical positive and negative terms respectively, and PMI(t,ti) is the pointwise mutual information (Lin, 1998b) between the terms t and ti.
A wide range of contextual information, such as surrounding words (Lowe and McDonald, 2000; Curran and Moens, 2002a), dependency or case structure (Hindle, 1990; Ruge, 1997; Lin, 1998), and dependency path (Lin and Pantel, 2001; Pado and Lapata, 2007), has been utilized for similarity calculation, and achieved considerable success.
Thus we rank each sense wsi WSw using Prevalence Score wsi = (11)  njNw dssnj  wnss(wsi,nj) wsiWSw wnss(wsi,nj) where the WordNet similarity score (wnss) is defined as: wnss(wsi,nj)= max nsxNSnj (wnss(wsi,nsx)) 2.2 Building the Thesaurus The thesaurus was acquired using the method described by Lin (1998).
1 Introduction Word alignment is a critical component in training statistical machine translation systems and has received a significant amount of research, for example, (Brown et al., 1993; Ittycheriah and Roukos, 2005; Fraser and Marcu, 2007), including work leveraging syntactic parse trees, e.g., (Cherry and Lin, 2006; DeNero and Klein, 2007; Fossum et al., 2008).
They propose two modifications to f-measure: varying the precision/recall tradeoff, and fully-connecting the alignment links before computing f-measure.11 Weighted Fully-Connected F-Measure Given a hypothesized set of alignment links H and a goldstandard set of alignment links G, we define H+ = fullyConnect(H) and G+ = fullyConnect(G), and then compute: f-measure(H+) = 1 precision(H+) + 1 recall(H+) For phrase-based Chinese-English and ArabicEnglish translation tasks, (Fraser and Marcu, 2007a) obtain the closest correlation between weighted fully-connected alignment f-measure and BLEU score using =0.5 and =0.1, respectively.
Probabilistic generative models like IBM 1-5 (Brown et al., 1993), HMM (Vogel et al., 1996), ITG (Wu, 1997), and LEAF (Fraser and Marcu, 2007) define formulas for P(f | e) or P(e, f), with ok-voon ororok sprok at-voon bichat dat erok sprok izok hihok ghirok totat dat arrat vat hilat ok-drubel ok-voon anok plok sprok at-drubel at-voon pippat rrat dat ok-voon anok drok brok jok at-voon krat pippat sat lat wiwok farok izok stok totat jjat quat cat lalok sprok izok jok stok wat dat krat quat cat lalok farok ororok lalok sprok izok enemok wat jjat bichat wat dat vat eneat lalok brok anok plok nok iat lat pippat rrat nnat wiwok nok izok kantok ok-yurp totat nnat quat oloat at-yurp lalok mok nok yorok ghirok clok wat nnat gat mat bat hilat lalok nok crrrok hihok yorok zanzanok wat nnat arrat mat zanzanat lalok rarok nok izok hihok mok wat nnat forat arrat vat gat Figure 1: Word alignment exercise (Knight, 1997).
There has been considerable skepticism over whether WSD will actually improve performance of applications, but we are now starting to see improvement in performance due to WSD in cross-lingual information retrieval (Clough and Stevenson, 2004; Vossen et al., 2006) and machine translation (Carpuat and Wu, 2007; Chan et al., 2007) and we hope that other applications such as question-answering, text simplication and summarisation might also benet as WSD methods improve.
As shown by McDonald and Nivre (2007), the Single Malt parser tends to suffer from two problems: error propagation due to the deterministic parsing strategy, typicallyaffectinglongdependenciesmorethan short ones, and low precision on dependencies originating in the artificial root node due to fragmented parses.9 The question is which of these problems is alleviatedbythemultipleviewsgivenbythecomponent parsers in the Blended system.
Looking rst at learning times, it is obvious that learning time depends primarily on the number of training instances, which is why we can observe a difference of several orders of magnitude in learning time between the biggest training set (Czech) and the smallest training set (Slovene) 14 This is shown by Nivre and Scholz (2004) in comparison to the iterative, arc-standard algorithm of Yamada and Matsumoto (2003) and by McDonald and Nivre (2007) in comparison to the spanning tree algorithm of McDonald, Lerman, and Pereira (2006).
c 2009 Association for Computational Linguistics Reverse Revision and Linear Tree Combination for Dependency Parsing Giuseppe Attardi Dipartimento di Informatica Universit`a di Pisa Pisa, Italy attardi@di.unipi.it Felice DellOrletta Dipartimento di Informatica Universit`a di Pisa Pisa, Italy felice.dellorletta@di.unipi.it 1 Introduction Deterministic transition-based Shift/Reduce dependency parsers make often mistakes in the analysis of long span dependencies (McDonald & Nivre, 2007).
The approach has been shown to give improvements over the MAP classifier in many areas of natural language processing including automatic speech recognition (Goel and Byrne, 2000), machine translation (Kumar and Byrne, 2004; Zhang and Gildea, 2008), bilingual word alignment (Kumar and Byrne, 2002), andparsing(Goodman, 1996; TitovandHenderson, 2006; Smith and Smith, 2007).
Note that it is straightforward to calculate these expected counts using a variant of the inside-outside algorithm (Baker, 1979) applied to the (Eisner, 1996) dependency-parsing data structures (Paskin, 2001) for projective dependency structures, or the matrix-tree theorem (Koo et al., 2007; Smith and Smith, 2007; McDonald and Satta, 2007) for nonprojective dependency structures.
FollowingJohnson(2007),Iusevariational Bayes EM (Beal, 2003) during the M-step for the transition distribution: l+1j|i = f(E[ni,j] +i)f(E[n i] + |C|i) (3) f(v) = exp((v)) (4) 60 (v) = braceleftBigg g(v 1 2) ifv> 7 (v+ 1)  1v o.w.
1 Introduction There has been a great deal of recent interest in the unsupervised discovery of syntactic structure from text, both parts-of-speech (Johnson, 2007; Goldwater and Griffiths, 2007; Biemann, 2006; Dasgupta and Ng, 2007) and deeper grammatical structure like constituency and dependency trees (Klein and Manning, 2004; Smith, 2006; Bod, 2006; Seginer, 2007; Van Zaanen, 2001).
For an HMM with a set of states T and a set of output symbols V : t  T t  Dir(1,|T|) (1) t  T t  Dir(1,|V |) (2) ti|ti1, ti1  Multi(ti1) (3) wi|ti, ti  Multi(ti) (4) One advantage of the Bayesian approach is that the prior allows us to bias learning toward sparser structures, by setting the Dirichlet hyperparameters , to a value less than one (Johnson, 2007; Goldwater and Griffiths, 2007).
prime 1 1 1 0.5 0.5 1 0.5 0.5 0.1 0.1 0.1 0.0001 0.0001 0.1 0.0001 0.0001 Further, we ran each setting of each estimator at least 10 times (from randomly jittered initial starting points) for at least 1,000 iterations, as Johnson (2007) showed that some estimators require many iterations to converge.
Given the parameters{pi0,pi,,K}of the HMM, the joint distribution over hidden states s and observationsy can be written (with s0 = 0): p(s,y|pi0,pi,,K) = Tproductdisplay t=1 p(st|st1)p(yt|st) As Johnson (2007) clearly explained, training the HMM with EM leads to poor results in PoS tagging.
Measures of cross-language relatedness are useful for a large number of applications, including cross-language information retrieval (Nie et al., 1999; Monz and Dorr, 2005), cross-language text classification (Gliozzo and Strapparava, 2006), lexical choice in machine translation (Och and Ney, 2000; Bangalore et al., 2007), induction of translation lexicons (Schafer and Yarowsky, 2002), cross-language annotation and resource projections to a second language (Riloff et al., 2002; Hwa et al., 2002; Mohammad et al., 2007).
0  500  1000  1500  2000  5000  10000  15000  20000  25000  30000 Number of interlanguage links Vector length aren ares arro enes enro esro Figure 5: Number of interlanguage links vs. vector length for the Miller-Charles data set  0  500  1000  1500  2000  2500  3000  3500  4000  5000  10000  15000  20000  25000  30000 Number of interlanguage links Vector length aren ares arro enes enro esro Figure 6: Number of interlanguage links vs. vector length for the WordSimilarity-353 data set edge bases (Lesk, 1986; Wu and Palmer, 1994; Resnik, 1995; Jiang and Conrath, 1997; Hughes and Ramage, 2007) or on large corpora (Salton et al., 1997; Landauer et al., 1998; Turney, 2001; Gabrilovich and Markovitch, 2007).
Method Source Spearman (Strube and Ponzetto, 2006) Wikipedia 0.190.48 (Jarmasz, 2003) WordNet 0.330.35 (Jarmasz, 2003) Rogets 0.55 (Hughes and Ramage, 2007) WordNet 0.55 (Finkelstein et al., 2002) Web corpus, WN 0.56 (Gabrilovich and Markovitch, 2007) ODP 0.65 (Gabrilovich and Markovitch, 2007) Wikipedia 0.75 SVM Web corpus, WN 0.78 Table 9: Comparison with previous work for WordSim353.
The techniques used to solve this problem can be roughly classified into two main categories: those relying on pre-existing knowledge resources (thesauri, semantic networks, taxonomies or encyclopedias) (Alvarez and Lim, 2007; Yang and Powers, 2005; Hughes and Ramage, 2007) and those inducing distributional properties of words from corpora (Sahami and Heilman, 2006; Chen et al., 2006; Bollegala et al., 2007).
4 Semantic Class Induction from Wikipedia Wikipedia has recently been used as a knowledge source for various language processing tasks, including taxonomy construction (Ponzetto and Strube, 2007a), coreference resolution (Ponzetto and Strube, 2007b), and English NER (e.g., Bunescu and Pasca (2006), Cucerzan (2007), Kazama and Torisawa (2007), Watanabe et al.
It has been used for a variety of tasks, such as wide-coverage parsing (Hockenmaier and Steedman, 2002; Clark and Curran, 2007), sentence realization (White, 2006), learning semantic parsers (Zettlemoyer and Collins, 2007), dialog systems (Kruijff et al., 2007), grammar engineering (Beavers, 2004; Baldridge et al., 2007), and modeling syntactic priming (Reitter et al., 2006).
available): SCISSOR (Ge and Mooney, 2005), an integrated syntactic-semantic parser; KRISP (Kate and Mooney, 2006), an SVM-based parser using string kernels; WASP (Wong and Mooney, 2006; Wong and Mooney, 2007), a system based on synchronous grammars; Z&C (Zettlemoyer and Collins, 2007)3, a probabilistic parser based on relaxed CCG grammars; and LU (Lu et al., 2008), a generative model with discriminative reranking.
Recently, methods from nonparametric Bayesian statistics have been gaining popularity as a way to approach unsupervised learning for a variety of tasks, including language modeling, word and morpheme segmentation, parsing, and machine translation (Teh et al., 2006; Goldwater et al., 2006a; Goldwater et al., 2006b; Liang et al., 2007; Finkel et al., 2007; DeNero et al., 2008).