waashk/dblp huggingface dataset:
a subcategorization acquisition system for french verbs. this paper presents a system capable of automatically acquiring subcategorization frames (scfs) for french verbs from the analysis of large corpora. we applied the system to a large newspaper corpus (consisting of 10 years of the french newspaper 'le monde') and acquired subcategorization information for 3267 verbs. the system learned 286 scf types for these verbs. from the analysis of 25 representative verbs, we obtained 0.82 precision, 0.59 recall and 0.69 f-measure. these results are comparable with those reported in recent related work.
inferring activity time in news through event modeling. many applications in nlp, such as question-answering and summarization, either require or would greatly benefit from the knowledge of when an event occurred. creating an effective algorithm for identifying the activity time of an event in news is difficult in part because of the sparsity of explicit temporal expressions. this paper describes a domain-independent machine-learning based approach to assign activity times to events in news. we demonstrate that by applying topic models to text, we are able to cluster sentences that describe the same event, and utilize the temporal information within these event clusters to infer activity times for all sentences. experimental evidence suggests that this is a promising approach, given evaluations performed on three distinct news article sets against the baseline of assigning the publication date. our approach achieves 90%, 88.7%, and 68.7% accuracy, respectively, outperforming the baseline twice.
arabic language modeling with finite state transducers. in morphologically rich languages such as arabic, the abundance of word forms resulting from increased morpheme combinations is significantly greater than for languages with fewer inflected forms (kirchhoff et al., 2006). this exacerbates the out-of-vocabulary (oov) problem. test set words are more likely to be unknown, limiting the effectiveness of the model. the goal of this study is to use the regularities of arabic inflectional morphology to reduce the oov problem in that language. we hope that success in this task will result in a decrease in word error rate in arabic automatic speech recognition.
an integraged architecture for generating parenthetical constructions. the aim of this research is to provide a principled account of the generation of embedded constructions (called parentheticals) and to implement the results in a natural language generation system. parenthetical constructions are frequently used in texts written in a good writing style and have an important role in text understanding. we propose a framework to model the rhetorical properties of parentheticals based on a corpus study and develop a unified natural language generation architecture which integrates syntax, semantics, rhetorical and document structure into a complex representation, which can be easily extended to handle parentheticals.
impact of initiative on collaborative problem solving. even though collaboration in peer learning has been shown to have a positive impact for students, there has been little research into collaborative peer learning dialogues. we analyze such dialogues in order to derive a model of knowledge co-construction that incorporates initiative and the balance of initiative. this model will be embedded in an artificial agent that will collaborate with students.
a supervised learning approach to automatic synonym identification based on distributional features. distributional similarity has been widely used to capture the semantic relatedness of words in many nlp tasks. however, various parameters such as similarity measures must be hand-tuned to make it work effectively. instead, we propose a novel approach to synonym identification based on supervised learning and distributional features, which correspond to the commonality of individual context types shared by word pairs. considering the integration with pattern-based features, we have built and compared five synonym classifiers. the evaluation experiment has shown a dramatic performance increase of over 120% on the f-1 measure basis, compared to the conventional similarity-based classification. on the other hand, the pattern-based features have appeared almost redundant.
an unsupervised vector approach to biomedical term disambiguation: integrating umls and medline. this paper introduces an unsupervised vector approach to disambiguate words in biomedical text that can be applied to all-word disambiguation. we explore using contextual information from the unified medical language system (umls) to describe the possible senses of a word. we experiment with automatically creating individualized stoplists to help reduce the noise in our dataset. we compare our results to senseclusters and humphrey et al. (2006) using the nlm-wsd dataset and with senseclusters using conflated data from the 2005 medline baseline.
combining source and target language information for name tagging of machine translation output. a named entity recognizer (ner) generally has worse performance on machine translated text, because of the poor syntax of the mt output and other errors in the translation. as some tagging distinctions are clearer in the source, and some in the target, we tried to integrate the tag information from both source and target to improve target language tagging performance, especially recall. in our experiments with chinese-to-english mt output, we first used a simple merge of the outputs from an et (entity translation) system and an english ner system, getting an absolute gain of 7.15% in f-measure, from 73.53% to 80.68%. we then trained an memm module to integrate them more discriminatively, and got a further average gain of 2.74% in f-measure, from 80.68% to 83.42%.
the role of positive feedback in intelligent tutoring systems. the focus of this study is positive feedback in one-on-one tutoring, its computational modeling, and its application to the design of more effective intelligent tutoring systems. a data collection of tutoring sessions in the domain of basic computer science data structures has been carried out. a methodology based on multiple regression is proposed, and some preliminary results are presented. a prototype intelligent tutoring system on linked lists has been developed and deployed in a collegelevel computer science class.
a re-examination on features in regression based approach to automatic mt evaluation. machine learning methods have been extensively employed in developing mt evaluation metrics and several studies show that it can help to achieve a better correlation with human assessments. adopting the regression svm framework, this paper discusses the linguistic motivated feature formulation strategy. we argue that "blind" combination of available features does not yield a general metrics with high correlation rate with human assessments. instead, certain simple intuitive features serve better in establishing the regression svm evaluation model. with six features selected, we show evidences to support our view through a few experiments in this paper.
a hierarchical approach to encoding medical concepts for clinical notes. this paper proposes a hierarchical text categorization (tc) approach to encoding free-text clinical notes with icd-9-cm codes. preliminary experimental result on the 2007 computational medicine challenge data shows a hierarchical tc system has achieved a micro-averaged f1 value of 86.6, which is comparable to the performance of state-of-the-art flat classification systems.
high frequency word entrainment in spoken dialogue. cognitive theories of dialogue hold that entrainment, the automatic alignment between dialogue partners at many levels of linguistic representation, is key to facilitating both production and comprehension in dialogue. in this paper we examine novel types of entrainment in two corpora---switchboard and the columbia games corpus. we examine entrainment in use of high-frequency words (the most common words in the corpus), and its association with dialogue naturalness and flow, as well as with task success. our results show that such entrainment is predictive of the perceived naturalness of dialogues and is significantly correlated with task success; in overall interaction flow, higher degrees of entrainment are associated with more overlaps and fewer interruptions.
coreference-inspired coherence modeling. research on coreference resolution and summarization has modeled the way entities are realized as concrete phrases in discourse. in particular there exist models of the noun phrase syntax used for discourse-new versus discourse-old referents, and models describing the likely distance between a pronoun and its antecedent. however, models of discourse coherence, as applied to information ordering tasks, have ignored these kinds of information. we apply a discourse-new classifier and pronoun coreference algorithm to the information ordering task, and show significant improvements in performance over the entity grid, a popular model of local coherence.
dialect classification for online podcasts fusing acoustic and language based structural and semantic information. the variation in speech due to dialect is a factor which significantly impacts speech system performance. in this study, we investigate effective methods of combining acoustic and language information to take advantage of (i) speaker based acoustic traits as well as (ii) content based word selection across the text sequence. for acoustics, a gmm based system is employed and for text based dialect classification, we proposed n-gram language models combined with latent semantic analysis (lsa) based dialect classifiers. the performance of the individual classifiers is established for the three dialect family case (dc rates vary from 69.1%--72.4%). the final combined system achieved a dc accuracy of 79.5% and significantly outperforms the baseline acoustic classifier with a relative improvement of 30%, confirming that an integrated dialect classification system is effective for american, british and australian dialects.
a linguistically annotated reordering model for btg-based statistical machine translation. in this paper, we propose a linguistically annotated reordering model for btg-based statistical machine translation. the model incorporates linguistic knowledge to predict orders for both syntactic and non-syntactic phrases. the linguistic knowledge is automatically learned from source-side parse trees through an annotation algorithm. we empirically demonstrate that the proposed model leads to a significant improvement of 1.55% in the bleu score over the baseline reordering model on the nist mt-05 chinese-to-english translation task.
novel semantic features for verb sense disambiguation. we propose a novel method for extracting semantic information about a verb's arguments and apply it to verb sense disambiguation (vsd). we contrast this method with two popular approaches to retrieving this information and show that it improves the performance of our vsd system and outperforms the other two approaches
correlation between rouge and human evaluation of extractive meeting summaries. automatic summarization evaluation is critical to the development of summarization systems. while rouge has been shown to correlate well with human evaluation for content match in text summarization, there are many characteristics in multiparty meeting domain, which may pose potential problems to rouge. in this paper, we carefully examine how well the rouge scores correlate with human evaluation for extractive meeting summarization. our experiments show that generally the correlation is rather low, but a significantly better correlation can be obtained by accounting for several unique meeting characteristics, such as disfluencies and speaker information, especially when evaluating system-generated summaries.
a unified syntactic model for parsing fluent and disfluent speech. this paper describes a syntactic representation for modeling speech repairs. this representation makes use of a right corner transform of syntax trees to produce a tree representation in which speech repairs require very few special syntax rules, making better use of training data. pcfgs trained on syntax trees using this model achieve high accuracy on the standard switchboard parsing task.
assessing the costs of sampling methods in active learning for annotation. traditional active learning (al) techniques assume that the annotation of each datum costs the same. this is not the case when annotating sequences; some sequences will take longer than others. we show that the al technique which performs best depends on how cost is measured. applying an hourly cost model based on the results of an annotation user study, we approximate the amount of time necessary to annotate a given sentence. this model allows us to evaluate the effectiveness of al sampling methods in terms of time spent in annotation. we acheive a 77% reduction in hours from a random baseline to achieve 96.5% tag accuracy on the penn treebank. more significantly, we make the case for measuring cost in assessing al methods.
recent improvements in the cmu large scale chinese-english smt system. in this paper we describe recent improvements to components and methods used in our statistical machine translation system for chinese-english used in the january 2008 gale evaluation. main improvements are results of consistent data processing, larger statistical models and a pos-based word reordering approach.
a novel feature-based approach to chinese entity relation extraction. relation extraction is the task of finding semantic relations between two entities from text. in this paper, we propose a novel feature-based chinese relation extraction approach that explicitly defines and explores nine positional structures between two entities. we also suggest some correction and inference mechanisms based on relation hierarchy and co-reference information etc. the approach is effective when evaluated on the ace 2005 chinese data set.
evaluating word prediction: framing keystroke savings. researchers typically evaluate word prediction using keystroke savings, however, this measure is not straightforward. we present several complications in computing keystroke savings which may affect interpretation and comparison of results. we address this problem by developing two gold standards as a frame for interpretation. these gold standards measure the maximum keystroke savings under two different approximations of an ideal language model. the gold standards additionally narrow the scope of deficiencies in a word prediction system.
computing confidence scores for all sub parse trees. computing confidence scores for applications, such as dialogue system, information retrieving and extraction, is an active research area. however, its focus has been primarily on computing word-, concept-, or utterance-level confidences. motivated by the need from sophisticated dialogue systems for more effective dialogs, we generalize the confidence annotation to all the subtrees, the first effort in this line of research. the other contribution of this work is that we incorporated novel long distance features to address challenges in computing multi-level confidence scores. using conditional maximum entropy (cme) classifier with all the selected features, we reached an annotation error rate of 26.0% in the swbd corpus, compared with a subtree error rate of 41.91%, a closely related benchmark with the charniak parser from (kahn et al., 2005).
beyond log-linear models: boosted minimum error rate training for n-best re-ranking. current re-ranking algorithms for machine translation rely on log-linear models, which have the potential problem of underfitting the training data. we present boostedmert, a novel boosting algorithm that uses minimum error rate training (mert) as a weak learner and builds a re-ranker far more expressive than log-linear models. boostedmert is easy to implement, inherits the efficient optimization properties of mert, and can quickly boost the bleu score on n-best re-ranking tasks. in this paper, we describe the general algorithm and present preliminary results on the iwslt 2007 arabic-english task.
choosing sense distinctions for wsd: psycholinguistic evidence. supervised word sense disambiguation requires training corpora that have been tagged with word senses, which begs the question of which word senses to tag with. the default choice has been wordnet, with its broad coverage and easy accessibility. however, concerns have been raised about the appropriateness of its fine-grained word senses for wsd. wsd systems have been far more successful in distinguishing coarsegrained senses than fine-grained ones (navigli, 2006), but does that approach neglect necessary meaning differences? recent psycholinguistic evidence seems to indicate that closely related word senses may be represented in the mental lexicon much like a single sense, whereas distantly related senses may be represented more like discrete entities. these results suggest that, for the purposes of wsd, closely related word senses can be clustered together into a more general sense with little meaning loss. the current paper will describe this psycholinguistic research and its implications for automatic word sense disambiguation.
the complexity of phrase alignment problems. many phrase alignment models operate over the combinatorial space of bijective phrase alignments. we prove that finding an optimal alignment in this space is np-hard, while computing alignment expectations is #p-hard. on the other hand, we show that the problem of finding an optimal alignment can be cast as an integer linear program, which provides a simple, declarative approach to viterbi inference for phrase alignment models that is empirically quite efficient.
four techniques for online handling of out-of-vocabulary words in arabic-english statistical machine translation. we present four techniques for online handling of out-of-vocabulary words in phrase-based statistical machine translation. the techniques use spelling expansion, morphological expansion, dictionary term expansion and proper name transliteration to reuse or extend a phrase table. we compare the performance of these techniques and combine them. our results show a consistent improvement over a state-of-the-art baseline in terms of bleu and a manual error analysis.
active learning with confidence. active learning is a machine learning approach to achieving high-accuracy with a small amount of labels by letting the learning algorithm choose instances to be labeled. most of previous approaches based on discriminative learning use the margin for choosing instances. we present a method for incorporating confidence into the margin by using a newly introduced online learning algorithm and show empirically that confidence improves active learning.
simulating the behaviour of older versus younger users when interacting with spoken dialogue systems. in this paper we build user simulations of older and younger adults using a corpus of interactions with a wizard-of-oz appointment scheduling system. we measure the quality of these models with standard metrics proposed in the literature. our results agree with predictions based on statistical analysis of the corpus and previous findings about the diversity of older people's behaviour. furthermore, our results show that these metrics can be a good predictor of the behaviour of different types of users, which provides evidence for the validity of current user simulation evaluation metrics.
mixture model pomdps for efficient handling of uncertainty in dialogue management. in spoken dialogue systems, partially observable markov decision processes (pomdps) provide a formal framework for making dialogue management decisions under uncertainty, but efficiency and interpretability considerations mean that most current statistical dialogue managers are only mdps. these mdp systems encode uncertainty explicitly in a single state representation. we formalise such mdp states in terms of distributions over pomdp states, and propose a new dialogue system architecture (mixture model pomdps) which uses mixtures of these distributions to efficiently represent uncertainty. we also provide initial evaluation results (with real users) for this architecture.
partial matching strategy for phrase-based statistical machine translation. this paper presents a partial matching strategy for phrase-based statistical machine translation (pbsmt). source phrases which do not appear in the training corpus can be translated by word substitution according to partially matched phrases. the advantage of this method is that it can alleviate the data sparseness problem if the amount of bilingual corpus is limited. we incorporate our approach into the state-of-the-art pbsmt system moses and achieve statistically significant improvements on both small and large corpora.
enforcing transitivity in coreference resolution. a desirable quality of a coreference resolution system is the ability to handle transitivity constraints, such that even if it places high likelihood on a particular mention being coreferent with each of two other mentions, it will also consider the likelihood of those two mentions being coreferent when making a final assignment. this is exactly the kind of constraint that integer linear programming (ilp) is ideal for, but, surprisingly, previous work applying ilp to coreference resolution has not encoded this type of constraint. we train a coreference classifier over pairs of mentions, and show how to encode this type of constraint on top of the probabilities output from our pairwise classifier to extract the most probable legal entity assignments. we present results on two commonly used datasets which show that enforcement of transitive closure consistently improves performance, including improvements of up to 3.6% using the b3 scorer, and up to 16.5% using cluster f-measure.
event matching using the transitive closure of dependency relations. this paper describes a novel event-matching strategy using features obtained from the transitive closure of dependency relations. the method yields a model capable of matching events with an f-measure of 66.5%.
extracting a representation from text for semantic analysis. we present a novel fine-grained semantic representation of text and an approach to constructing it. this representation is largely extractable by today's technologies and facilitates more detailed semantic analysis. we discuss the requirements driving the representation, suggest how it might be of value in the automated tutoring domain, and provide evidence of its validity.
speakers' intention prediction using statistics of multi-level features in a schedule management domain. speaker's intention prediction modules can be widely used as a pre-processor for reducing the search space of an automatic speech recognizer. they also can be used as a preprocessor for generating a proper sentence in a dialogue system. we propose a statistical model to predict speakers' intentions by using multi-level features. using the multi-level features (morpheme-level features, discourse-level features, and domain knowledge-level features), the proposed model predicts speakers' intentions that may be implicated in next utterances. in the experiments, the proposed model showed better performances (about 29% higher accuracies) than the previous model. based on the experiments, we found that the proposed multi-level features are very effective in speaker's intention prediction.