hrithikpiyush/acl-arc huggingface dataset:
PE-dev was used to obtain w * to define the utility model . This was done by MERT optimization ( Och , 2003 ) towards post-edits under the TER target metric . Note that the goal of our experi -
The largest lexical evaluation we know of is that of Schulte im Walde ( 2002b ) for German . She evaluates 3,000 German verbs with a token frequency between 10 and 2,000 against the Duden ( Dudenredaktion 2001 ) . We will refer to this work and the methods and results presented by Schulte im Walde again in Sections 6.2 and 6.3 .
The second facet is independent of the clinical task and pertains to the structure of a well-built clinical question . The following four components have been identified as the key elements of a question related to patient care ( Richardson et al. 1995 ) : • What is the primary problem or disease ?
As a generalization , Briscoe ( 2001 ) notes that lexicons such as COMLEX tend to demonstrate high precision but low recall . Briscoe and Carroll ( 1997 ) report on manually analyzing an open-class vocabulary of 35,000 head words for predicate subcategorization information and comparing the results against the subcategorization details in COMLEX . Precision was quite high ( 95 % ) , but recall was low ( 84 % ) .
The highest accuracy is obtained by SL-DOP at 12 n 14 : an LP of 90.8 % and an LR of 90.7 % . This is roughly an 11 % relative reduction in error rate over Charniak ( 2000 ) and Bods PCFG-reduction reported in Table 1 . Compared to the reranking technique in Collins ( 2000 ) , who obtained an LP of 89.9 % and an LR of 89.6 % , our results show a 9 % relative error rate reduction .
Louwerse et al. ( 2006 ) and Louwerse et al. ( 2007 ) study the relation between eye gaze , facial expression , pauses and dialogue structure in annotated English map-task dialogues ( Anderson et al. , 1991 ) and find correlations between the various modalities both within and across speakers . Finally , feedback expressions ( head nods and shakes ) are successfully predicted from speech , prosody and eye gaze in interaction with Embodied Communication Agents as well as human communication ( Fujie et al. , 2004 ; Morency et al. , 2005 ; Morency et al. , 2007 ; Morency et al. , 2009 ) . Our work is in line with these studies , all of which focus on the relation between linguistic expressions , prosody , dialogue content and gestures .
We now turn to our second type of error analysis , the evaluation of the grammaticality of the parse trees in terms of gender and number agreement patterns . We use the agreement checker code developed by Alkuhlani and Habash ( 2011 ) and evaluate our baseline ( MaltParser using only CORE12 ) , best performing model ( Easy-First Parser using CORE12 + DET+LMM+PERSON+FN * NGR g + p ) , and the gold reference . The agreement checker verifies , for all verb -- nominal subject relations and noun -- adjective relations found in the tree , whether the agreement conditions are met or not .
The knowledge base is used to check the factual correctness of the answers first , and then a diagnoser checks the explanation correctness . The diagnoser , based on Dzikovska et al. ( 2008b ) , outputs a diagnosis which consists of lists of correct , contradictory and non-mentioned objects and relations from the student 's answer . At present , the system uses a heuristic matching algorithm to classify relations into the appropriate category , though in the future we may consider a classifier similar to Nielsen et al. ( 2008 ) .
One thus needs to distinguish the lexical rule specification provided by the linguist from the fully explicit lexical rule relations integrated into the theory . The formalization of DLRs provided by Meurers ( 1995 ) defines a formal lexical rule specification language and provides a semantics for that language in two steps : A rewrite system enriches the lexical rule specification into a fully explicit description of the kind shown in Figure 1 . This description can then be given the standard set-theoretical interpretation of King ( 1989 , 1994 ) . '
Similar to our previous work ( Chan and Ng , 2005b ) , we used the supervised WSD approach described in ( Lee and Ng , 2002 ) for our experiments , using the naive Bayes algorithm as our classifier . Knowledge sources used include partsof-speech , surrounding words , and local collocations .
Similar to ( Li et al. , 2013a ) , our summarization system is , which consists of three key components : an initial sentence pre-selection module to select some important sentence candidates ; the above compression model to generate n-best compressions for each sentence ; and then an ILP summarization method to select the best summary sentences from the multiple compressed sentences . The sentence pre-selection model is a simple supervised support vector regression ( SVR ) model that predicts a salience score for each sentence and selects the top ranked sentences for further processing ( compression and summarization ) .
Xia ( 1999 ) also presents a similar method for the extraction of a TAG from the Penn Treebank . The extraction procedure consists of three steps : First , the bracketing of the trees in the Penn Treebank is corrected and extended based on the approaches of Magerman ( 1994 ) and Collins ( 1997 ) . Then the elementary trees are read off in a quite straightforward manner .
• Transition-based : An implementation of the transition-based dependency parsing framework ( Nivre , 2008 ) using an arc-eager transition strategy and are trained using the perceptron algorithm as in Zhang and Clark ( 2008 ) with a beam size of 8 . Beams with varying sizes can be used to produce k-best lists .
• Mean Average Precision ( MAP ) is the average of precision values after each relevant document is retrieved ( Baeza-Yates and Ribeiro-Neto 1999 ) . It is the most widely accepted single-value metric in information retrieval , and is seen to balance the need for both precision and recall .
Furthermore , the availability of rich ontological resources , in the form of the Unified Medical Language System ( UMLS ) ( Lindberg et al. , 1993 ) , and the availability of software that leverages this knowledge -- MetaMap ( Aronson , 2001 ) for concept identification and SemRep ( Rindflesch and Fiszman , 2003 ) for relation extraction -- provide a foundation for studying the role of semantics in various tasks . McKnight and Srinivasan ( 2003 ) have previously examined the task of categorizing sentences in medical abstracts using supervised discriminative machine learning techniques . Building on the work of Ruch et al. ( 2003 ) in the same domain , we present a generative approach that attempts to directly model the discourse structure of MEDLINE abstracts using Hidden Markov Models ( HMMs ) ; cfXXX ( Barzilay and Lee , 2004 ) .
Acoustic models for HTK is trained with the continuous speech database of the Acoustical Society of Japan ( Kobayashi et al. , 1992 ) . This recognizer incrementally outputs word hypotheses as soon as they are found in the best-scored path in the forward search ( Hirasawa et al. , 1998 ) using the ISTAR ( Incremental Structure Transmitter And Receiver ) protocol , which conveys word graph information as well as word hypotheses . This incremental output allows the language understanding module to process recognition results before the speech interval ends , and thus real-time responses are possible .
The recent great advances in speech and language technologies have made it possible to build fully implemented spoken dialogue systems ( Aust et al. , 1995 ; Allen et al. , 1996 ; Zue et al. , 2000 ; Walker et al. , 2000 ) . One of the next research goals is to make these systems task-portable , that is , to simplify the process of porting to another task domain .
16 In principle , we could have used a classification method to predict clusters from the values of the confidence measures for unseen cases . We posit that this would not have a significant effect on the results , in particular for MML-based classification techniques , such as Decision Graphs ( Oliver 1993 ) . comparative evaluation in Section 4 .
Although Bod 's method obtains very competitive results on the Wall Street Journal ( WSJ ) task , the parsing time was reported to be over 200 seconds per sentence ( Bod 2003 ) . Collins & Duffy ( 2002 ) showed how the perceptron algorithm can be used to efficiently compute the best parse with DOP1 's subtrees , reporting a 5.1 % relative reduction in error rate over the model in Collins ( 1999 ) on the WSJ . Goodman ( 2002 ) furthermore showed how Bonnema et al. 's ( 1999 ) and Bod 's ( 2001 ) estimators can be incorporated in his PCFGreduction , but did not report any experiments with these reductions .
All modules other than the last one are related to the conversion process from LTAG into HPSG , and the last one enables to obtain LTAG analysis from the obtained HPSG analysis . Tateisi et al. also translated LTAG into HPSG ( Tateisi et al. , 1998 ) . However , their method depended on translator 's intuitive analysis of the original grammar .
The system utilizes several large size biological databases including three NCBI databases ( GenPept [ 11 ] , RefSeq [ 12 ] , and Entrez GENE [ 13 ] ) , PSD database from Protein Information Resources ( PIR ) [ 14 ] , and Proceedings of the ACL Interactive Poster and Demonstration Sessions ,
Our Moses systems use default settings . The LM uses the monolingual data and is trained as a five-gram9 using the SRILM-Toolkit ( Stolcke , 2002 ) . We run MERT separately for each system .
Politically-oriented text Sentiment analysis has specifically been proposed as a key enabling technology in eRulemaking , allowing the automatic analysis of the opinions that people submit ( Shulman et al. , 2005 ; Cardie et al. , 2006 ; Kwon et al. , 2006 ) . There has also been work focused upon determining the political leaning ( e.g. , `` liberal '' vs. `` conservative '' ) of a document or author , where most previously-proposed methods make no direct use of relationships between the documents to be classified ( the `` unlabeled '' texts ) ( Laver et al. , 2003 ; Efron , 2004 ; Mullen and Malouf , 2006 ) . An exception is Grefenstette et al. ( 2004 ) , who experimented with determining the political orientation of websites essentially by classifying the concatenation of all the documents found on that site .
First of all , iteration would increase the complexity of building a model of a paragraph ; infinite iteration would almost certainly make impossible such a construction in real time . Secondly , the cooperative principle of Grice ( 1975 , 1978 ) , under the assumption that referential levels of a writer and a reader are quite similar , implies that the writer should structure the text in a way that makes the construction of his intended model easy for the reader ; and this seems to imply that he should appeal only to the most direct knowledge of the reader . Finally , it has been shown by Groesser ( 1981 ) that the ratio of derived to explicit information necessary for understanding a piece of text is about 8:1 ; furthermore , our reading of the analysis of five paragraphs by Crothers ( 1979 ) strongly suggests that only the most direct or obvious inferences are being made in the process of building a model or constructing a theory of a paragraph .
However , we can achieve this with two weights : ( 1 , 1 ) for f1 and ( − 1 , 1 ) for f2 . In this paper , inspired by KNN-SVM ( Zhang et al. , 2006 ) , we propose a local training method , which trains sentence-wise weights instead of a single weight , to address the above two problems . Compared with global training methods , such as MERT , in which training and testing are separated , our method works in an online fashion , in which training is performed during testing .
Most of these endeavours have focused on purely statistical acquisition techniques ( Church and Hanks , ` However , our interpretation of LFs in this work is much looser , since we admitted verbs that would not be considered to be members of true collocations as Mel ` cuk et al. ( 1984 1999 ) define them , i.e. groups of lexical units that share a restricted cooccurrence relationship . 1990 ) , on linguisitic acquisition ( by the use of Part-of-Speech filters hand-crafted by a linguist ) ( Oueslati , 1999 ) or , more frequently , on a combination of the two ( Smadja , 1993 ; Kilgarriff and Tugwell , 2001 , for example ) . It is worth noting that although these techniques are able to identify N-V pairs , they do not specify the relationship between N and V , nor are they capable of focusing on a subset of N-V pairs .
In some cases rankings may work well enough . Samuelsson and Voutilainen ( 1997 ) report excellent part-of-speech tagging results using a handcrafted approach that is close to OT .3 More speculatively , imagine an OT grammar for stylistic revision of parsed sentences . The tension between preserving the original author 's text ( faithfulness to the underlying form ) and making it readable in various ways ( well-formedness ) is right up OT 's alley .
The subject is instead specified externally in the matrix phrase : The judge wants [ XCOMP to open an inquiry ] . While many linguistic theories state subcategorization requirements in terms of phrase structure ( CFG categories ) , Dalrymple ( 2001 ) questions the viability and universality of such an approach because of the variety of ways in which grammatical functions may be realized at the language-specific constituent structure level . LFG argues that subcategorization requirements are best stated at the f-structure level , in functional rather than phrasal terms .
At this point it may be proper to comment on the relationship between our theory of coherence and theories advocated by others . We are going to make such a comparison with the theories proposed by J. Hobbs ( 1979 , 1982 ) that represent a more computationally oriented approach to coherence , and those of T.A. van Dijk and W. Kintch ( 1983 ) , who are more interested in addressing psychological and cognitive aspects of discourse coherence . The quoted works seem to be good representatives for each of the directions ; they also point to related literature .
We only use one sample to extract the translation grammar because multiple samples would result in a grammar that would be too large . 11 From ( Zollmann and Vogel , 2011 ) , we find that the performance of SAMT system is similar with the method of labeling SCFG rules with POS tags . Thus , to be convenient , we only conduct experiments with the SAMT system .
( Michiels ( 1982 ) contains further description and discussion of LDOCE . ) In this paper we focus on the exploitation of the LDOCE grammar coding system ; Alshawi et al. ( 1985 ) and Alshawi ( 1987 ) describe further research in Cambridge utilising different types of information available in LDOCE . The information available in the dictionary is both very rich and diverse , but also typically only semiformalised , as it is intended for human , rather than machine , interpetation .
Thus , they were not supervised during the experiment . Gurevych ( 2006 ) observed that some annotators were not familiar with the exact definition of semantic relatedness . Their results differed particularly in cases of antonymy or distributionally related pairs .
2 Lexical resources for TE and CLTE All current approaches to monolingual TE , either syntactically oriented ( Rus et al. , 2005 ) , or applying logical inference ( Tatu and Moldovan , 2005 ) , or adopting transformation-based techniques ( Kouleykov and Magnini , 2005 ; Bar-Haim et al. , 2008 ) , incorporate different types of lexical knowledge to support textual inference . Such information ranges from i ) lexical paraphrases ( textual equivalences between terms ) to ii ) lexical relations preserving entailment between words , and iii ) wordlevel similarity/relatedness scores .
Cimino and Mendonc ¸ a reported good performance for etiology , diagnosis , and in particular therapy , but not prognosis . Although originally developed as a tool to assist in query formulation , Booth ( 2000 ) pointed out that PICO frames can be employed to structure IR results for improving precision . PICO-based querying in information retrieval is merely an instance of faceted querying , which has been widely used by librarians since the introduction of automated retrieval systems ( e.g. , Meadow et al. 1989 ) .
There has been some controversy , at least for simple stemmers ( Lovins , 1968 ; Porter , 1980 ) , about the effectiveness of morphological analysis for document retrieval ( Harman , 1991 ; Krovetz , 1993 ; Hull , 1996 ) . The key for quality improvement seems to be rooted mainly in the presence or absence of some form of dictionary .
The probability of the input sentence is a constant across all the candidate derivations , so we only need to find the most probable derivation . in history-based models ( Black et al. , 1993 ) , the probability estimate for each derivation decision di is conditioned on the previous derivation decisions d1 , ... , d , _ 1 , which is called the derivation history at step i . This allows us to use the chain rule for conditional probabilities to derive the probability of the entire derivation as the multiplication of the probabilities for each of its decisions .
This technique closely resembles the off-line constraint propagation technique described by Marriott , Naish , and Lassez ( 1988 ) . The reader is referred to Meurers and Minnen ( 1996 ) for a more detailed discussion of our use of constraint propagation .32 We illustrate the result of constraint propagation with our example grammar . Since the running example of this paper was kept small , for expository reasons , by only including features that do get changed by one of the lexical rules ( which violates the empirical observation mentioned above ) , the full set of lexical rules would not provide a good example .
The availability of toolkits for this weighted case ( Mohri et al. , 1998 ; van Noord and Gerdemann , 2001 ) promises to unify much of statistical NLP . Such tools make it easy to run most current approaches to statistical markup , chunking , normalization , segmentation , alignment , and noisy-channel decoding , ' including classic models for speech recognition ( Pereira and Riley , 1997 ) and machine translation ( Knight and Al-Onaizan , 1998 ) . Moreover , once the models are expressed in the finitestate framework , it is easy to use operators to tweak them , to apply them to speech lattices or other sets , and to combine them with linguistic resources .
The automation of help-desk responses has been previously tackled using mainly knowledge-intensive paradigms , such as expert systems ( Barr and Tessler 1995 ) and case-based reasoning ( Watson 1997 ) . Such technologies require significant human input , and are difficult to create and maintain ( Delic and Lahaix 1998 ) . In contrast , the techniques examined in this article are corpus-based and data-driven .
To formalize the notion of what it means for a category to be more `` plausible '' , we extend the category generator of our previous work , which we will call PCAT . We can define PCAT using a probabilistic grammar ( Garrette et al. , 2014 ) . The grammar may first generate a start or end category ( ( S ) , ( E ) ) with probability pse or a special tokendeletion category ( ( D ) ; explained in § 5 ) with probability pdel , or a standard CCG category C :
derivational and inflectional process , most prepositions , conjunctions , pronouns , and possessive forms are attached to the Arabic surface word . It is these orthographic variations and complex morphological structure that make Arabic language processing challenging ( Xu et al. , 2001 ; Xu et al. , 2002 ) . Both tasks are performed with a statistical framework : the mention detection system is similar to the one presented in ( Florian et al. , 2004 ) and the coreference resolution system is similar to the one described in ( Luo et al. , 2004 ) .
Both can be formally defined and they reflect different levels of shallow parsing patterns . The first is the one used in the chunking competition in CoNLL-2000 ( Tjong Kim Sang and Buchholz , 2000 ) . In this case , a full parse tree is represented in a flat form , producing a representation as in the example above .
Strongly equivalent grammars enable the sharing of ideas developed in each formalism . There have been many studies on parsing techniques ( Poller and Becker , 1998 ; Flickinger et al. , 2000 ) , ones on disambiguation models ( Chiang , 2000 ; Kanayama et al. , 2000 ) , and ones on programming/grammar-development environ - 1In this paper , we use the term LTAG to refer to FBLTAG , if not confusing .
Although there is a trend towards analysis of full article texts , we believe that abstracts still provide a tremendous amount of information , and much value can still be extracted from them . For example , Gay et al. ( 2005 ) experimented with abstracts and full article texts in the task of automatically generating index term recommendations and discovered that using full article texts yields at most a 7.4 % improvement in F-score . Demner-Fushman et al. ( 2005 ) found a correlation between the quality and strength of clinical conclusions in the full article texts and abstracts .
The terms have been identified as the most specific to our corpus by a program developed by Drouin ( 2003 ) and called TER1vloSTAT . The ten most specific nouns have been produced by comparing our corpus of computing to the French corpus Le Monde , composed of newspaper articles ( Lemay et al. , 2004 ) . Note that to prevent any bias in the results , none of these terms were used as positive examples during the pattern inference step .
From a computational linguistic view , the framework will also need to take into account the granularity of the unit ( for example , POS tagging requires sentence-units , but anaphoric annotation needs paragraphs or larger ) . Secondly , we need to investigate techniques for identifying identical documents , virtually identical documents and highly repetitive documents , such as those pioneered by Fletcher ( 2004b ) and shingling techniques described by Chakrabarti ( 2002 ) . The second stage of our work will involve implementing the framework within a P2P environment .
merge and ii ) how to merge . Following the work of Stymne and Cancedda ( 2011 ) , we implement a linear-chain CRF merging system using the following features : stemmed ( separated ) surface form , part-of-speech14 and frequencies from the training corpus for bigrams/merging of word and word +1 , word as true prefix , word +1 as true suffix , plus frequency comparisons of these . The CRF is trained on the split monolingual data .
Ushioda et al. ( 1993 ) employ an additional statistical method based on log-linear models and Bayes ' theorem to filter the extra noise introduced by the parser and were the first to induce relative frequencies for the extracted frames . Manning ( 1993 ) attempts to improve on the approach of Brent ( 1993 ) by passing raw text through a stochastic tagger and a finite-state parser ( which includes a set of simple rules for subcategorization frame recognition ) in order to extract verbs and the constituents with which they co-occur . He assumes 19 different subcategorization
This is particularly true for the medical domain . From an IR view , a lot of specialized research has already been carried out for medical applications , with emphasis on the lexico-semantic aspects of dederivation and decomposition ( Pacak et al. , 1980 ; Norton and Pacak , 1983 ; Wolff , 1984 ; Wingert , 1985 ; Dujols et al. , 1991 ; Baud et al. , 1998 ) . While one may argue that single-word compounds are quite rare in English ( which is not the case in the medical domain either ) , this is certainly not true for German and other basically agglutinative languages known for excessive single-word nominal compounding .