hrithikpiyush/acl-arc huggingface dataset:
Our error analysis also revealed some interesting issues associated with using an external standard such as COMLEX . In the future , we hope to evaluate the automatic annotations and extracted lexicon against Propbank ( Kingsbury and Palmer 2002 ) . Apart from the related approach of Miyao , Ninomiya , and Tsujii ( 2004 ) , which does not distinguish between argument and adjunct prepositional phrases , our treebank and automatic f-structure annotation-based architecture for the automatic acquisition of detailed subcategorization frames is quite unlike any of the architectures presented in the literature .
The primary resource gathered is a `` phrasal lexicon , '' constructed by extracting over 200,000 phrases from the Penn Treebank and having them translated into French by three Web-based machine translation ( MT ) systems . Each set of translations is stored separately , and for each set the `` marker hypothesis '' ( Green 1979 ) is used to segment the phrasal lexicon into a `` marker lexicon . '' The marker hypothesis is a universal psycholinguistic constraint which states that natural languages are '' marked '' for complex syntactic structure at surface form by a closed set of specific lexemes and morphemes .
One way to increase the precision of the mapping process is to impose some linguistic constraints on the sequences such as simple noun-phrase contraints ( Gaussier , 1995 ; Kupiec , 1993 ; hua Chen and Chen , 94 ; Fung , 1995 ; Evans and Zhai , 1996 ) . It is also possible to focus on non-compositional compounds , a key point in bilingual applications ( Su et al. , 1994 ; Melamed , 1997 ; Lin , 99 ) .
The idea of using preferences among theories is new , hence it was described in more detail . `` Coherence , '' as outlined above , can be understood as a declarative ( or static ) version of marker passing ( Hirst 1987 ; Charniak 1983 ) , with one difference : the activation spreads to theories that share a predicate , not through the IS-A hierarchy , and is limited to elementary facts about predicates appearing in the text . The metalevel rules we are going to discuss in Section 6 , and that deal with the Gricean maxims and the meaning of `` but , '' can be easily expressed in the languages of set theory or higher order logic , but not everything expressible in those languages makes sense in natural language .
In addition to headwords , dictionary search through the pronunciation field is available ; Carter ( 1987 ) has merged information from the pronunciation and hyphenation fields , creating an enhanced phonological representation which allows access to entries by broad phonetic class and syllable structure ( Huttenlocher and Zue , 1983 ) . In addition , a fully flexible access system allows the retrieval of dictionary entries on the basis of constraints specifying any combination of phonetic , lexical , syntactic , and semantic information ( Boguraev et al. , 1987 ) . Independently , random selection of dictionary entries is also provided to allow the testing of software on an unbiased sample .
But unlike the cache model , it uses a multipass strategy . Clarkson and Robinson ( 1997 ) developed a way of incorporating standard n-grams into the cache model , using mixtures of language models and also exponentially decaying the weight for the cache prediction depending on the recency of the word 's last occurrence .
This is particularly important in NLP because of the high redundancy across tasks and approaches . Machine learning methods should be interchangeable : Transformation-based learning ( TBL ) ( Brill , 1993 ) and Memory-based learning ( MBL ) ( Daelemans et al. , 2002 ) have been applied to many different problems , so a single interchangeable component should be used to represent each method . We will base these components on the design of Weka ( Witten and Frank , 1999 ) .
It is not clear , in particular , that the rules for computing semantic types for verbs are well enough motivated linguistically or that the LDOCE lexicographers were sensitive enough to the different transformational potential of the various classes of verbs to make a rule such as our one for Object Raising viable . We tested the classification of verbs into semantic types using a verb list of 139 pre-classified items drawn from the lists published in Rosenbaum ( 1967 ) and Stockwell et al. ( 1973 ) . Figure 16 gives the number of verbs classified under each category by these authors and the number successfully classified into the same categories by the system .
Dialogue Act Modeling sequence with the highest posterior probability : The combination of likelihood and prior modeling , HMMs , and Viterbi decoding is fundamentally the same as the standard probabilistic approaches to speech recognition ( Bahl , Jelinek , and Mercer 1983 ) and tagging ( Church 1988 ) . It maximizes the probability of getting the entire DA sequence correct , but it does not necessarily find the DA sequence that has the most DA labels correct ( Dermatas and Kokkinakis 1995 ) .
Current practise elsewhere includes the distribution of URL lists , but given the dynamic nature of the web , this is not sufficiently robust . Other solutions such as complete caching of the corpora are not typically adopted due to legal concerns over copyright and redistribution of web data , issues considered at length by Fletcher ( 2004a ) . Other requirements for reference corpora such as retrieval and storage of metadata for web pages are beyond the scope of what we propose here .
The full formulation of the integrated grammar transformation and construction of the finite automaton is rather long and is therefore not given here . A very similar formulation , for another grammar transformation , is given in Nederhof ( 1998 ) .
The development set is NIST02 evaluation data , and the test datasets are NIST05 , NIST06 , and NIST08 . We run GIZA + + ( Och and Ney , 2000 ) on the training corpus in both directions ( Koehn et al. , 2003 ) to obtain the word alignment for each sentence pair . We train a 4-gram language model on the Xinhua portion of the English Gigaword corpus using the SRILM Toolkits ( Stolcke , 2002 ) with modified Kneser-Ney smoothing ( Chen and Goodman , 1998 ) .
An example of this is the estimation of maximum entropy models , from simple iterative estimation algorithms used by Ratnaparkhi ( 1998 ) that converge very slowly , to complex techniques from the optimisation literature that converge much more rapidly ( Malouf , 2002 ) . Other attempts to address efficiency include the fast Transformation Based Learning ( TBL ) Toolkit ( Ngai and Florian , 2001 ) which dramatically speeds up training TBL systems , and the translation of TBL rules into finite state machines for very fast tagging ( Roche and Schabes , 1997 ) . The TNT POS tagger ( Brants , 2000 ) has also been designed to train and run very quickly , tagging between 30,000 and 60,000 words per second .
Clearly , the above characterization is very wide ranging . Consequently , fusion has been applied to a wide variety of pattern recognition and decision theoretic problems -- using a plethora of theories , techniques , and tools -- including some applications in computational linguistics ( e.g. , Brill and Wu 1998 ; van Halteren , Zavrel , and Daelemans 1998 ) and speech technology ( e.g. , Bowles and Damper 1989 ; Romary and Pierre11989 ) . According to Abbott ( 1999 , 290 ) , `` While the reasons [ that ] combining models works so well are not rigorously understood , there is ample evidence that improvements over single models are typical ... .
( to pronounced / tu / ) When it comes to sentence-level prosody , especially phrasing , it is often true , as we will see below , that a sequence of words dominated by the same syntactic node cohere more closely than a sequence of words dominated by two different nodes . This observation has led some researchers , e.g. , Cooper and Paccia-Cooper ( 1980 ) , to claim a direct mapping between the syntactic phrase and the prosodic phrase . However , this claim is controversial because of the misa : ignments that occur between the two levels of phrasing .
In our previous work ( Zhang and Chai , 2009 ) , conversation entailment is formulated as the following : given a conversation segment D which is represented by a set of clauses D = d1 ∧ ... ∧ dm , and a hypothesis H represented by another set of clauses H = h1 ∧ ... ∧ hn , the prediction on whether D entails H is determined by the product of probabilities that each hypothesis clause hj is entailed from all the conversation segment clauses d1 ... dm as follows . This is based on a simple assumption that whether a clause is entailed from a conversation segment is conditionally independent from other clauses .
They complemented this approach with machine learning techniques that automatically learn the weights of different retrieval models . Berger et al. ( 2000 ) compared two retrieval approaches ( TF.IDF and query expansion ) and two predictive approaches ( statistical translation and latent variable models ) . Jijkoun and de Rijke ( 2005 ) compared different variants of retrieval techniques .
Interaction between components is coordinated by the dialogue manager which uses the informationstate approach ( Larsson and Traum , 2000 ) . The dialogue state is represented by a cumulative answer analysis which tracks , over multiple turns , the correct , incorrect , and not-yet-mentioned parts 1Other factors such as student confidence could be considered as well ( Callaway et al. , 2007 ) . of the answer .
Another drawback to using an existing external gold standard such as COMLEX to evaluate an automatically induced subcategorization lexicon is that the resources are not necessarily constructed from the same source data . As noted above , it is well documented ( Roland and Jurafsky 1998 ) that subcategorization frames ( and their frequencies ) vary across domains . We have extracted frames from two sources ( the WSJ and the Brown corpus ) , whereas COMLEX was built using examples from the San Jose Mercury News , the Brown corpus , several literary works from the Library of America , scientific abstracts from the U.S. Department of Energy , and the WSJ .
As these formalisms are fully lexicalized with an invariant ( LTAG and CCG ) or limited ( HPSG ) rule component , the extraction of a lexicon essentially amounts to the creation of a grammar . Chen and Vijay-Shanker ( 2000 ) explore a number of related approaches to the extraction of a lexicalized TAG from the Penn-II Treebank with the aim of constructing a statistical model for parsing . The extraction procedure utilizes a head percolation table as introduced by Magerman ( 1995 ) in combination with a variation of Collins 's ( 1997 ) approach to the differentiation between complement and adjunct .
Compared with left-to-right transduction , middle-out transduction also aids robustness because , when complete derivations are not available , partial derivations tend to have meaningful headwords . At the same time , we believe our method has advantages over the approach developed initially at IBM ( Brown et al. 1990 ; Brown et al. 1993 ) for training translation systems automatically . One advantage is that our method attempts to model the natural decomposition of sentences into phrases .
McKnight and Srinivasan ( 2003 ) have previously examined the task of categorizing sentences in medical abstracts using supervised discriminative machine learning techniques . Building on the work of Ruch et al. ( 2003 ) in the same domain , we present a generative approach that attempts to directly model the discourse structure of MEDLINE abstracts using Hidden Markov Models ( HMMs ) ; cfXXX ( Barzilay and Lee , 2004 ) . Although our results were not obtained from the same exact collection as those used by authors of these two previous studies , comparable experiments suggest that our techniques are competitive in terms of performance , and may offer additional advantages as well .
A dictionary access process is fired off , which dynamically constructs a search tree and navigates through it from a given homograph directly to the offset in the lispified file from where all the associated information can be retrieved . As Alshawi ( 1987 ) points out , given that no situations were envisaged where the information from the tape would be altered once installed in secondary storage , this simple and convenComputational Linguistics , Volume 13 , Numbers 3-4 , July-December 1987 205 Bran Boguraev and Ted Briscoe Large Lexicons for Natural Language Processing tional access strategy is perfectly adequate . The use of such standard database indexing techniques makes it possible for an active dictionary process to be very undemanding with respect to main memory utilisation .
Differently , we require that each multi-word non-terminal node must have two child nodes . This is because the binary structure has been verified to be very effective for tree-based translation ( Wang et al. , 2007 ; Zhang et al. , 2011a ) . The generation process starts at root node N .
For some experiments we also measure the standard intrinsic parser metrics unlabeled attachment score ( UAS ) and labeled attachment score ( LAS ) ( Buchholz and Marsi , 2006 ) . In terms of treebank data , the primary training corpus is the Penn Wall Street Journal Treebank ( PTB ) ( Marcus et al. , 1993 ) .
In our system , in some cases the smallest chunk obtainable via the marker-based segmentation process may be something like ( 27 ) : ( 27 ) <DET> the good man : le bon homme In such cases , if our system were confronted with a good man , it would not be able to translate such a phrase , assuming this to be missing from the marker lexicon . Accordingly , we convert examples such as ( 27 ) into their generalized equivalents , as in ( 28 ) : ( 28 ) <DET> good man : bon homme That is , where Block ( 2000 ) substitutes variables for various words in his templates , we replace certain lexical items with their marker tag . Given that examples such as '` <DET> a : un '' are likely to exist in the word-level lexicon , they may be inserted at the point indicated by the marker tag to form the correct translation un bon homme .
The comparison that must be made to determine which option is better in a given situation is how often the first will error correct incorrectly as opposed to how much error correcting power we will lose by using the second . How it is done is beyond the scope of this paper but is explained in detail in Fink ( 1983 ) . The Merge function takes two inputs , M1 and M2 , which have been determined by the Mergeable function to be similar in some way by considering their respective environments and meanings .
These tools currently train in less than 10 minutes on the standard training materials and tag faster than TNT , the fastest existing POS tagger . These tools use a highly optimised GIS implementation and provide sophisticated Gaussian smoothing ( Chen and Rosenfeld , 1999 ) . We expect even faster training times when we move to conjugate gradient methods .
Architecture ( Section 6 ) . The inference rules that were necessary to convert one list of properties into another do not sit comfortably within the received NLG pipeline model ( e.g. , Reiter and Dale 2000 ) . An example of such an inference rule is the one that transforms a list of the form ( mouse , > 10 cm ) into one of the form ( mouse , size ( x ) = max2 ) if only two mice are larger than 10 cm .
A simple example of this is the case assignment involved in building a prepositional phrase , i.e. , prepositions must assign case , and DPs much receive case . Niyogi ( 2001 ) has developed an agenda-driven chart parser for the feature-driven formalism described above ; please refer to his paper for a description of the parsing algorithm . I have adapted it for my needs and developed grammar fragments that reflect my non-lexicalist semantic framework .
In addition to a referring function , noun phrases ( NP ) can also serve communicative goals such as providing new information about the referent and expressing the speaker 's emotional attitude towards the referent ( Appelt , 1985 ; Jordan , 2000 ) . In Example ( 1 ) below , the part in italics refers to an object in a museum , and the part in boldface provides additional information about it .
The list of semantic relations with which we work is based on extensive literature study ( Barker et al. , 1997a ) . Three lists of relations for three syntactic levels -- inter-clause , intra-clause ( case ) and nounmodifier relations -- were next combined based on syntactic and semantic phenomena .
Demner-Fushman et al. ( 2005 ) found a correlation between the quality and strength of clinical conclusions in the full article texts and abstracts . This paper presents experiments with generative content models for analyzing the discourse structure of medical abstracts , which has been confirmed to follow the four-section pattern discussed above ( Salanger-Meyer , 1990 ) . For a variety of reasons , medicine is an interesting domain of research .
These predictions enable our system to recommend a particular method for handling a new ( unseen ) request ( Marom , Zukerman , and Japkowicz 2007 ) . Following Lekakos and Giaglis ( 2007 ) , one approach for achieving this objective consists of applying supervised learning , where a winning method is selected for each case in the training set , all the training cases are labeled accordingly , and then the system is trained to predict a winner for unseen cases . However , in our situation , there is not always one single winner ( two methods can perform similarly well for a given request ) , and there are different ways to pick winners ( for example , based on F-score or precision ) .
To recap , it has the following MaltParser attributes ( machine learning features ) : 4 word-form attributes , 7 POS tag attributes , and 5 deprel attributes ( some of which are not useful for the Nivre `` eager '' algorithm ) , totaling 16 attributes and two more for every new feature described in Section 4.3 and on ( e.g. , CASE ) . 11 Nivre ( 2008 ) reports that non-projective and pseudo-projective algorithms outperform the `` eager '' projective algorithm in MaltParser , but our training data did not contain any non-projective dependencies . The Nivre `` standard '' algorithm is also reported there to do better on Arabic , but in a preliminary experimentation , it did slightly worse than the `` eager '' one , perhaps due to the high percentage of right branching ( left headed structures ) in our Arabic training set -- an observation already noted in Nivre ( 2008 ) .
It would be worthwhile to investigate applying some of the more sophisticated ensemble learning techniques which have been proposed in the literature ( Dietterich , 1997 ) . In particular , boosting ( Schapire , 1999 ; Abney et al. , 1999 ) offers the possibility of achieving high accuracy from a collection of classifiers which individually perform quite poorly .
alignment . For MT the most commonly used heuristic is called grow diagonal final ( Och and Ney 2003 ) . This starts with the intersection of the sets of aligned points and adds points around the diagonal that are in the union of the two sets of aligned points .
We report results with two metrics : ECM-F and ACEValue . ECM-F is an entity-constrained mention Fmeasure ( cfXXX ( Luo et al. , 2004 ) for how ECM-F is computed ) , and ACE-Value is the official ACE evaluation metric . The result is shown in Table 4 : the baseline numbers without stem features are listed under `` Base , '' and the results of the coreference system with stem features are listed under `` Base + Stem . ''
This methodology is well-described in the literature ( Landauer , et al. , 1998 ; Manning and Schütze , 1999 ) . In order to obtain semantic representations of each word , we apply our previous strategy ( Schone and Jurafsky ( 2000 ) ) . Rather than using a termdocument matrix , we had followed an approach akin to that of Schütze ( 1993 ) , who performed SVD on a Nx2N term-term matrix .
This setup is also scalable to a higher number of word pairs ( 350 ) as was shown in Gurevych ( 2006 ) . Finkelstein et al. ( 2002 ) annotated a larger set of word pairs ( 353 ) , too . They used a 0-10 range of relatedness scores , but did not give further details about their experimental setup .
User satisfaction was assessed by asking the subjects to fill out a questionnaire after interacting with each version of the system . Furthermore , a number of performance features , largely based on the PARADISE dialogue evaluation scheme ( Walker et al. , 1997 ) , were automatically logged , derived , or manually annotated . In addition , we logged the cues automatically detected in each user utterance , as well as the initiative distribution for each turn and the dialogue acts selected to generate each system response .
If neither order appears in the training data , then neither a b nor b a and an order must be randomly assigned . Shaw and Hatzivassiloglou ( 1999 ) propose to generalize the direct evidence method so that it can apply to unseen pairs of adjectives by computing the transitive closure of the ordering relation . That is , if a c and c b , we can conclude that a b. To take an example from the BNC , the adjectives large and green never occur together in the training data , and so would be assigned a random order by the direct evidence method .
We evaluated our translations with IBM 's BLEU evaluation metric ( Papineni et al. , 2001 ) , using the same evaluation method and reference retranslations that were used for evaluation at HLT Workshop 2002 at CLSP ( Haji 6 et al. , 2002 ) . We used four reference retranslations of 490 sentences selected from the WSJ sections 22 , 23 , and 24 , which were themselves used as the fifth reference .
Such a statement is data-dependent in the sense that if the model satisfies the constraints then we do not need to change it ; if it is far from satisfying it we might need to make very dramatic changes . PR is closely related to the work of Mann and McCallum ( 2007 , 2008 ) , who concurrently developed the idea of using penalties based on posterior expectations of features to guide semi-supervised learning . They call their method generalized expectation ( GE ) constraints or alternatively expectation regularization .
We experimented with various Weka classifiers , comprising Hidden Naive Bayes , SMO , ID3 , LADTree and Decision Table . The best results on most of our data were obtained using Hidden Naive Bayes ( HNB ) ( Zhang et al. , 2005 ) . Therefore , here we show the results of this classifier .
All rules from the rule-based system are incorporated into mlSystem ruleFeats as the features . mlSystem ruleFeats + atomFeats We augment mlSystem ruleFeats with more features from our previous work ( Markert et al. , 2012 ; Hou et al. , 2013a ; Hou et al. , 2013b ) on bridging anaphora recognition and antecedent selection . Some of these features overlap with the atomic features used in the rule-based system .
This is similar to `` one sense per collocation '' idea of Yarowsky ( 1993 ) . The description of the EAGLE workbench for linguistic engineering ( Baldwin et al. 1997 ) mentions a case normalization module that uses a heuristic in which a capitalized word in an ambiguous position should be rewritten without capitalization if it is found lower-cased in the same document . This heuristic also employs a database of bigrams and unigrams of lower-cased and capitalized words found in unambiguous positions .
There has also been work focused upon determining the political leaning ( e.g. , `` liberal '' vs. `` conservative '' ) of a document or author , where most previously-proposed methods make no direct use of relationships between the documents to be classified ( the `` unlabeled '' texts ) ( Laver et al. , 2003 ; Efron , 2004 ; Mullen and Malouf , 2006 ) . An exception is Grefenstette et al. ( 2004 ) , who experimented with determining the political orientation of websites essentially by classifying the concatenation of all the documents found on that site . Others have applied the NLP technologies of near-duplicate detection and topic-based text categorization to politically oriented text ( Yang and Callan , 2005 ; Purpura and Hillard , 2006 ) .