hrithikpiyush/acl-arc huggingface dataset:
This correlation coefficient is an upper bound of performance for automatic SR measures applied on the same dataset . Resnik ( 1995 ) reported a correlation of r = .9026.10 The results are not directly comparable , because he only used noun-noun pairs , words instead of concepts , a much smaller dataset , and measured semantic similarity instead of semantic relatedness . Finkelstein et al. ( 2002 ) did not report inter-subject correlation for their larger dataset .
( Taft , 1975 ) with his experiment on English inflected words , argued that lexical decision responses of polymorphemic words depends upon the base word frequency . Similar observation for surface word frequency was also observed by ( Bertram et al. , 2000 ; Bradley , 1980 ; Burani et al. , 1987 ; Burani et al. , 1984 ; Schreuder et al. , 1997 ; Taft 1975 ; Taft , 2004 ) where it has been claimed that words having low surface frequency tends to decompose . Later , Baayen ( 2000 ) proposed the dual processing race model that proposes that a specific morphologically complex form is accessed via its parts if the frequency of that word is above a certain threshold of frequency , then the direct route will win , and the word will be accessed as a whole .
MT system combination ( Matusov , Ueffing , and Ney 2006 ) . But their importance has grown far beyond machine translation : for instance , transferring annotations between languages ( Yarowsky and Ngai 2001 ; Hwa et al. 2005 ; Ganchev , Gillenwater , and Taskar 2009 ) ; discovery of paraphrases ( Bannard and Callison-Burch 2005 ) ; and joint unsupervised POS and parser induction across languages ( Snyder and Barzilay 2008 ) . IBM Models 1 and 2 and the HMM are simple and tractable probabilistic models , which produce the target sentence one target word at a time by choosing a source word and generating its translation .
Relationships between the unlabeled items Carvalho and Cohen ( 2005 ) consider sequential relations between different types of emails ( e.g. , between requests and satisfactions thereof ) to classify messages , and thus also explicitly exploit the structure of conversations . Previous sentiment-analysis work in different domains has considered inter-document similarity ( Agarwal and Bhattacharyya , 2005 ; Pang and Lee , 2005 ; Goldberg and Zhu , 2006 ) or explicit inter-document references in the form of hyperlinks ( Agrawal et al. , 2003 ) .
Exactly how to get from the parse tree to an appropriate meaning representation is a current research topic in our group . However , the method we are currently using in the ATIS domain ( Seneff et al. 1991 ) represents our most promising approach to this problem . We have decided to limit semantic frame types to a small set of choices such as CLAUSE ( for a sentence-level concept , such as request ) , PREDICATE ( for a functional operation ) , REFERENCE ( essentially proper noun ) , and QSET ( for a set of objects ) .
From a more pessimistic one , an author may feel to have lost power in this openness . Henceforth the collaborative traits of blogs and wikis ( McNeill , 2005 ) emphasize annotation , comment , and strong editing . They give more power to readers , eventually filling the gap the so-called active readers become authors as well .
2.1.2 The ICA Approach The ICA system ( Hepple , 2000 ) aims to reduce the training time by introducing independence assumptions on the training samples that dramatically reduce the training time with the possible downside of sacrificing performance . To achieve the speedup , the ICA system disallows any interaction between the learned rules , by enforcing the following two assumptions :
One of the next research goals is to make these systems task-portable , that is , to simplify the process of porting to another task domain . To this end , several toolkits for building spoken dialogue systems have been developed ( Barnett and Singh , 1997 ; Sasajima et al. , 1999 ) . One is the CSLU Toolkit ( Sutton et al. , 1998 ) , which enables rapid prototyping of a spoken dialogue system that incorporates a finite-state dialogue model .
While earlier work in this direction concentrated on manual construction of rules , most of the recent work has been motivated by the observation that shallow syntactic information can be extracted using local information by examining the pattern itself , its nearby context and the local part-of-speech information . Thus , over the past few years , along with advances in the use of learning and statistical methods for acquisition of full parsers ( Collins , 1997 ; Charniak , 1997a ; Charniak , 1997b ; Ratnaparkhi , 1997 ) , significant progress has been made on the use of statistical learning methods to recognize shallow parsing patterns syntactic phrases or words that participate in a syntactic relationship ( Church , 1988 ; Ramshaw and Marcus , 1995 ; Argamon et al. , 1998 ; Cardie and Pierce , 1998 ; Munoz et al. , 1999 ; Punyakanok and Roth , 2001 ; Buchholz et al. , 1999 ; Tjong Kim Sang and Buchholz , 2000 ) . Research on shallow parsing was inspired by psycholinguistics arguments ( Gee and Grosjean , 1983 ) that suggest that in many scenarios ( e.g. , conversational ) full parsing is not a realistic strategy for sentence processing and analysis , and was further motivated by several arguments from a natural language engineering viewpoint .
Note that from an experimental point of view , this is a very convenient problem to work with because we can automatically determine ground truth ( and thus avoid the need for manual annotation ) simply by consulting publicly available voting records . Task properties Determining whether or not a speaker supports a proposal falls within the realm of sentiment analysis , an extremely active research area devoted to the computational treatment of subjective or opinion-oriented language ( early work includes Wiebe and Rapaport ( 1988 ) , Hearst ( 1992 ) , Sack ( 1994 ) , and Wiebe ( 1994 ) ; see Esuli ( 2006 ) for an active bibliography ) . In particular , since we treat each individual speech within a debate as a single `` document '' , we are considering a version of document-level sentiment-polarity classification , namely , automatically distinguishing between positive and negative documents ( Das and Chen , 2001 ; Pang et al. , 2002 ; Turney , 2002 ; Dave et al. , 2003 ) .
Various approaches for computing semantic relatedness of words or concepts have been proposed , e.g. dictionary-based ( Lesk , 1986 ) , ontology-based ( Wu and Palmer , 1994 ; Leacock and Chodorow , 1998 ) , information-based ( Resnik , 1995 ; Jiang and Conrath , 1997 ) or distributional ( Weeds and Weir , 2005 ) . The knowledge sources used for computing relatedness can be as different as dictionaries , ontologies or large corpora .
It is these orthographic variations and complex morphological structure that make Arabic language processing challenging ( Xu et al. , 2001 ; Xu et al. , 2002 ) . Both tasks are performed with a statistical framework : the mention detection system is similar to the one presented in ( Florian et al. , 2004 ) and the coreference resolution system is similar to the one described in ( Luo et al. , 2004 ) . Both systems are built around from the maximum-entropy technique ( Berger et al. , 1996 ) .
Other contexts , such as adjectival and nominal predicates , could also aid the prediction , but have not yet been investigated . The advantage of tuning similarity to the application of interest has been shown previously by Weeds and Weir ( 2005 ) . They optimize a few metaparameters separately for the tasks of thesaurus generation and pseudodisambiguation .
Although there are other discussions of the paragraph as a central element of discourse ( e.g. Chafe 1979 , Halliday and Hasan 1976 , Longacre 1979 , Haberlandt et al. 1980 ) , all of them share a certain limitation in their formal techniques for analyzing paragraph structure . Discourse linguists show little interest in making the structural descriptions precise enough so that a computational grammar of text could adapt them and use them .
Learning algorithms . We experiment with four learners commonly employed in language learning : Decision List ( DL ) : We use the DL learner as described in Collins and Singer ( 1999 ) , motivated by its success in the related tasks of word sense disambiguation ( Yarowsky , 1995 ) and NE classification ( Collins and Singer , 1999 ) . We apply add-one smoothing to smooth the class posteriors .
We offer a theorem that highlights the broad applicability of these modeling techniques .4 If f ( input , output ) is a weighted regular relation , then the following statements are equivalent : ( 1 ) f is a joint probabilistic relation ; ( 2 ) f can be computed by a Markovian FST that halts with probability 1 ; ( 3 ) f can be expressed as a probabilistic regexp , i.e. , a regexp built up from atomic expressions a : b ( for a E E U -LCB- E -RCB- , b E A U -LCB- E -RCB- ) using concatenation , probabilistic union + p , and probabilistic closure * p. For defining conditional relations , a good regexp language is unknown to us , but they can be defined in several other ways : ( 1 ) via FSTs as in Fig. 1c , ( 2 ) by compilation of weighted rewrite rules ( Mohri and Sproat , 1996 ) , ( 3 ) by compilation of decision trees ( Sproat and Riley , 1996 ) , ( 4 ) as a relation that performs contextual left-to-right replacement of input substrings by a smaller conditional relation ( Gerdemann and van Noord , 1999 ) ,5 ( 5 ) by conditionalization of a joint relation as discussed below . A central technique is to define a joint relation as a noisy-channel model , by composing a joint relation with a cascade of one or more conditional relations as in Fig. 1 ( Pereira and Riley , 1997 ; Knight and Graehl , 1998 ) . The general form is illustrated by 3Conceptually , the parameters represent the probabilities of reading another a ( A ) ; reading another b ( ν ) ; transducing b to p rather than q ( µ ) ; starting to transduce p to a rather than x ( p ) .
where ~ f ( · ) extracts a feature vector from a classified ~ document , θ are the corresponding weights of those features , and Zθ ( x ) def Ey u ( x , y ) is a normalizer . We use the same set of binary features as in previous work on this dataset ( Pang et al. , 2002 ; Pang and Lee , 2004 ; Zaidan et al. , 2007 ) . Specifically , let V = -LCB- v1 , ... , v177441 be the set of word types with count > 4 in the full 2000-document corpus .
As discussed in Section 1 , however , the conversational nature of our data implies the existence of various relationships that can be exploited to improve cumulative classification accuracy for speech segments belonging to the same debate . Our classification framework , directly inspired by Blum and Chawla ( 2001 ) , integrates both perspectives , optimizing its labeling of speech segments based on both individual speech-segment classification scores and preferences for groups of speech segments to receive the same label . In this section , we discuss the specific classification framework that we adopt and the set of mechanisms that we propose for modeling specific types of relationships .
Looking at Hebrew , a Semitic language related to Arabic , Tsarfaty and Sima'an ( 2007 ) report that extending POS and phrase structure tags with definiteness information helps unlexicalized PCFG parsing . As for work on Arabic ( MSA ) , results have been reported on the PATB ( Kulick , Gabbard , and Marcus 2006 ; Diab 2007 ; Green and Manning 2010 ) , the Prague Dependency Treebank ( PADT ) ( Buchholz and Marsi 2006 ; Nivre 2008 ) and the CATiB ( Habash and Roth 2009 ) . Recently , Green and Manning ( 2010 ) analyzed the PATB for annotation consistency , and introduced an enhanced split-state constituency grammar , including labels for short idafa constructions and verbal or equational clauses .
Although some SBD systems can be trained on relatively small sets of labeled examples , their performance in such cases is somewhat lower than their optimal performance . For instance , Palmer and Hearst ( 1997 ) report that the SATZ system ( decision tree variant ) was trained on a set of about 800 labeled periods , which corresponds to a corpus of about 16,000 words . This is a relatively small training set that can be manually marked in a few hours ' time .
On one side , we plan to explore alternative ways to build phrase and paraphrase tables . One possible direction is to consider linguistically motivated approaches , such as the extraction of syntactic phrase tables as proposed by ( Yamada and Knight , 2001 ) . Another interesting direction is to investigate the potential of paraphrase patterns ( i.e. patterns including partof-speech slots ) , extracted from bilingual parallel corpora with the method proposed by ( Zhao et al. , 2009 ) .
The first lexical substitution method was proposed by Chapman and Davida ( 1997 ) . Later works , such as Atallah et al. ( 2001a ) , Bolshakov ( 2004 ) , Taskiran et al. ( 2006 ) and Topkara et al. ( 2006b ) , further made use of part-ofspeech taggers and electronic dictionaries , such as WordNet and VerbNet , to increase the robustness of the method . Taskiran et al. ( 2006 ) attempt to use context by prioritizing the alternatives using an ngram language model ; that is , rather than randomly choose an option from the synonym set , the system relies on the language model to select the synonym .
A number of speech understanding systems have been developed during the past fifteen years ( Barnett et al. 1980 , Dixon and Martin 1979 , Erman et al. 1980 , Haton and Pierrel 1976 , Lea 1980 , Lowerre and Reddy 1980 , Medress 1980 , Reddy 1976 , Walker 1978 , and Wolf and Woods 1980 ) . Most of these efforts concentrated on the interaction between low level information sources from a speech recognizer and a natural language processor to discover the meaning of an input sentence .
The Tags model also does much better than the only other broad coverage neural network parser ( Costa et al. , 2001 ) . The bottom panel of table 1 lists the results for the chosen lexicalized model ( SSN-Freq > 200 ) and five recent statistical parsers ( Ratnaparkhi , 1999 ; Collins , 1999 ; Charniak , 2000 ; Collins , 2000 ; Bod , 2001 ) . The performance of the lexicalized model falls in the middle of this range , only being beaten by the three best current parsers , which all achieve equivalent performance .
Finally , since Python can produce stand-alone executables directly , it will be possible to create distributable code that does not require the entire infrastructure or Python interpreter to be installed . The basic Python reflection has already been implemented and used for large scale experiments with POS tagging , using pyMPI ( a message passing interface library for Python ) to coordinate experiments across a cluster of over 100 machines ( Curran and Clark , 2003 ; Clark et al. , 2003 ) . An example of using the Python tagger interface is shown in Figure 1 .
The correct translations of a word that has several correct translations will be assigned a lower probability than the correct translation of a word that has only one correct translation . This imbalance foils thresholding strategies , clever as they might be ( Gale & Church , 1991 ; Wu & Xia , 1994 ; Chen , 1996 ) . The likelihoods in the word-to-word model remain unnormalized , so they do not compete .
Training was done on the Penn Treebank ( Marcus et al. , 1993 ) Wall Street Journal data , sections 02-21 . To train the CSCL shallow parser we had first to convert the WSJ data to a flat format that directly provides the phrase annotations .
Wikipedia ( WIKI ) . We performed Latent Semantic Analysis ( LSA ) over Wikipedia using the jLSI tool ( Giuliano , 2007 ) to measure the relatedness between words in the dataset . Then , we filtered all the pairs with similarity lower than 0.7 as proposed by ( Kouylekov et al. , 2009 ) .
A third relevant line of research is on reusing bitexts between related languages without or with very little adaptation , which works well for very closely related languages . For example , our previous work ( Nakov and Ng , 2009 ; Nakov and Ng , 2012 ) experimented with various techniques for combining a small bi-text for a resource-poor language ( Indonesian or Spanish , pretending that Spanish is resource-poor ) with a much larger bi-text for a related resource-rich language ( Malay or Portuguese ) ; the target language of all bi-texts was English . However , our previous work did not attempt language adaptation , except for very simple transliteration for Portuguese -- Spanish that ignored context entirely ; since it could not substitute one word for a completely different word , it did not help much for Malay -- Indonesian , which use unified spelling .
Some efforts have tackled tasks such as automatic image caption generation ( Feng and Lapata , 2010a ; Ordonez et al. , 2011 ) , text illustration ( Joshi et al. , 2006 ) , or automatic location identification of Twitter users ( Eisenstein et al. , 2010 ; Wing and Baldridge , 2011 ; Roller et al. , 2012 ) . Another line of research approaches grounded language knowledge by augmenting distributional approaches of word meaning with perceptual information ( Andrews et al. , 2009 ; Steyvers , 2010 ; Feng and Lapata , 2010b ; Bruni et al. , 2011 ; Silberer and Lapata , 2012 ; Johns and Jones , 2012 ; Bruni et al. , 2012a ; Bruni et al. , 2012b ; Silberer et al. , 2013 ) . Although these approaches have differed in model definition , the general goal in this line of research has been to enhance word meaning with perceptual information in order to address one of the most common criticisms of distributional semantics : that the `` meaning of words is entirely given by other words '' ( Bruni et al. , 2012b ) .
Furthermore , semantic relatedness is an intuitive concept and being forced to assign fine-grained continuous values is felt to overstrain the test subjects . Gurevych ( 2005 ) replicated the experiment of Rubenstein and Goodenough with the original 65 word pairs translated into German . She used an adapted experimental setup where test subjects had to assign discrete values -LCB- 0,1,2,3,4 -RCB- and word pairs were presented in isolation .
The problem of generating ordered sequences of adjectives is an instance of the more general problem of selecting among a number of possible outputs from a natural language generation system . One approach to this more general problem , taken by the ` Nitrogen ' generator ( Langkilde and Knight , 1998a ; Langkilde and Knight , 1998b ) , takes advantage of standard statistical techniques by generating a lattice of all possible strings given a semantic representation as input and selecting the most likely output using a bigram language model . Langkilde and Knight report that this strategy yields good results for problems like generating verb/object collocations and for selecting the correct morphological form of a word .
2Thus , the difference to token n-grams is that the tokens of different type are removed from the streams , before the features are created . where mk is one mention in entity e , and the basic model building block PL ( L = 1 | e , mk , m ) is an exponential or maximum entropy model ( Berger et al. , 1996 ) . For the start model , we use the following approximation :
Table 5 presents the coverage and unique/best coverage of each method ( the percentage of queries covered only by this method or for which this method produces a better reply than other methods ) , and the average and standard deviation of the precision and F-score obtained by each method ( calculated over the requests that are covered ) . 13 We also employed sequence-based measures using the ROUGE tool set ( Lin and Hovy 2003 ) , with similar results to those obtained with the word-by-word measures . Performance of the different methods for each data set : ( a ) coverage , ( b ) F-score , and ( c ) precision .
This is not possible when modeling word sequences , where only the probability of a sentence is actually used in the HMM training . Second , using continuous distributions allows us to leverage a variety of tools ( e.g. , LDA ) that have been shown to be successful in other fields , such as speech recognition ( Evermann et al. , 2004 ) . Table 2 ( b ) represents the closest head-to-head comparison between our generative approach ( HMM with LDA ) and state-of-the-art results reported by M&S using SVMs .
In this section , we validate the contribution of key tag sets and morphological features -- and combinations thereof -- using a different parser : the Easy-First Parser ( Goldberg and Elhadad 2010 ) . As in Section 4 , all models are evaluated on both gold and non-gold ( machine-predicted ) feature values .
The major drawback of this approach , however , is the tremendous amount of redundancy in the lexicon -- for example , the class of prototypical transitive verbs where the agent appears as the subject and the theme as the direct object must all duplicate this pattern . The typical solution to the redundancy problem is to group verbs according to their argument realization patterns ( Levin , 1993 ) , possibly arranged in an inheritance hierarchy . The argument structure and syntax-tosemantics mapping would then only need to be specified once for each verb class .
Adding selectional restrictions ( semantic feature information , Hobbs 1977 ) does not solve the problem , because isolated features offer only part of the background knowledge necessary for reference disambiguation . Later , Hobbs ( 1979 , 1982 ) proposed a knowledge base in which information about language and the world would be encoded , and he emphasized the need for using `` salience '' in choosing facts from this knowledge base . We will investigate the possibility that the structure of this knowledge base can actually resemble the structure of , for example , natural language dictionaries .
Transliteration , a technique that guesses the likely translations of a word based on pronunciation , can be readily used in translating proper nouns . Another technique is automatic discovery of translations from parallel or non-parallel corpora ( Fung and Mckeown , 1997 ) . Since traditional lexicons are more or less static repositories of knowledge , techniques that discover translation from newly published materials can supplement them with corpus-specific vocabularies .
The method used for the acquisition of N-V pairs relies mainly on ASARES , a pattern inference tool . ASARES is presented in detail in ( Claveau et al. , 2003 ) . We simply give a short account of its basic principles herein .
1 . Opposition ( called `` adversative '' or `` contrary-to-expectation '' by Halliday and Hasan 1976 ; cfXXX also Quirk et al. 1972 , p. 672 ) . The ship arrived but the passengers could not get off .
A number of applications have relied on distributional analysis ( Harris , 1971 ) in order to build classes of semantically related terms . This approach , which uses words that appear in the context of terms to formulate hypotheses on their semantic relatedness ( Habert et al. , 1996 , for example ) , does not specify the relationship itself .
We go beyond previous work , however , and explore additional lexical and inflectional features . Previous work with MaltParser in Russian , Turkish , and Hindi showed gains with CASE but not with agreement features ( Eryigit , Nivre , and Oflazer 2008 ; Nivre , Boguslavsky , and Iomdin 2008 ; Nivre 2009 ) . Our work is the first to show gains using agreement in MaltParser and in Arabic dependency parsing , and the first to use functional features for this task .
written as fully specified relations between words , rather , only what is supposed to be changed is specified . Consider , for example , the lexical rule in Figure 2 , which encodes a passive lexical rule like the one presented by Pollard and Sag ( 1987 , 215 ) in terms of the setup of Pollard and Sag ( 1994 , ch . 9 ) .