waashk/dblp huggingface dataset:
unsupervised learning of acoustic sub-word units. accurate unsupervised learning of phonemes of a language directly from speech is demonstrated via an algorithm for joint unsupervised learning of the topology and parameters of a hidden markov model (hmm); states and short state-sequences through this hmm correspond to the learnt sub-word units. the algorithm, originally proposed for unsupervised learning of allophonic variations within a given phoneme set, has been adapted to learn without any knowledge of the phonemes. an evaluation methodology is also proposed, whereby the state-sequence that aligns to a test utterance is transduced in an automatic manner to a phoneme-sequence and compared to its manual transcription. over 85% phoneme recognition accuracy is demonstrated for speaker-dependent learning from fluent, large-vocabulary speech.
text segmentation with lda-based fisher kernel. in this paper we propose a domain-independent text segmentation method, which consists of three components. latent dirichlet allocation (lda) is employed to compute words semantic distribution, and we measure semantic similarity by the fisher kernel. finally global best segmentation is achieved by dynamic programming. experiments on chinese data sets with the technique show it can be effective. introducing latent semantic information, our algorithm is robust on irregular-sized segments.
multi-domain sentiment classification. this paper addresses a new task in sentiment classification, called multi-domain sentiment classification, that aims to improve performance through fusing training data from multiple domains. to achieve this, we propose two approaches of fusion, feature-level and classifier-level, to use training data from multiple domains simultaneously. experimental studies show that multi-domain sentiment classification using the classifier-level approach performs much better than single domain classification (using the training data individually).
pairwise document similarity in large collections with mapreduce. this paper presents a mapreduce algorithm for computing pairwise document similarity in large document collections. mapreduce is an attractive framework because it allows us to decompose the inner products involved in computing document similarity into separate multiplication and summation stages in a way that is well matched to efficient disk access patterns across several machines. on a collection consisting of approximately 900,000 newswire articles, our algorithm exhibits linear growth in running time and space in terms of the number of documents.
fastsum: fast and accurate query-based multi-document summarization. we present a fast query-based multi-document summarizer called fastsum based solely on word-frequency features of clusters, documents and topics. summary sentences are ranked by a regression svm. the summarizer does not use any expensive nlp techniques such as parsing, tagging of names or even part of speech information. still, the achieved accuracy is comparable to the best systems presented in recent academic competitions (i.e., document understanding conference (duc)). because of a detailed feature analysis using least angle regression (lars), fastsum can rely on a minimal set of features leading to fast processing times: 1250 news documents in 60 seconds.
an adaptive approach to collecting multimodal input. multimodal dialogue systems allow users to input information in multiple modalities. these systems can handle simultaneous or sequential composite multimodal input. different coordination schemes require such systems to capture, collect and integrate user input in different modalities, and then respond to a joint interpretation. we performed a study to understand the variability of input in multimodal dialogue systems and to evaluate methods to perform the collection of input information. an enhancement in the form of incorporation of a dynamic time window to a multimodal input fusion module was proposed in the study. we found that the enhanced module provides superior temporal characteristics and robustness when compared to previous methods.
extraction and verification of ko-ou expressions from large corpora. in the japanese language, as a predicate is placed at the end of a sentence, the content of a sentence cannot be inferred until reaching the end. however, when the content is complicated and the sentence is long, people want to know at an earlier stage in the sentence whether the content is negative, affirmative, or interrogative. in japanese, the grammatical form called the ko-ou relation exists. the ko-ou relation is a kind of concord. if a ko element appears, then an ou element appears in the latter part of a sentence. it is being pointed out that the ko-ou relation gives advance notice to the element that appears in the latter part of a sentence. in this paper, we present the method of extracting automatically the ko-ou expression data from large-scale electronic corpus and verify the usefulness of the ko-ou expression data.
a speech interface for open-domain question-answering. speech interfaces to question-answering systems offer significant potential for finding information with phones and mobile networked devices. we describe a demonstration of spoken question answering using a commercial dictation engine whose language models we have customized to questions, a web-based text-prediction interface allowing quick correction of errors, and an open-domain question-answering system, answerbus, which is freely available on the web. we describe a small evaluation of the effect of recognition errors on the precision of the answers returned and make some concrete recommendations for modifying a question-answering system for improving robustness to spoken input.
high-precision identification of discourse new and unique noun phrases. coreference resolution systems usually attempt to find a suitable antecedent for (almost) every noun phrase. recent studies, however, show that many definite nps are not anaphoric. the same claim, obviously, holds for the indefinites as well.in this study we try to learn automatically two classifications, &plusmn;discourse_new and &plusmn;unique, relevant for this problem. we use a small training corpus (muc-7), but also acquire some data from the internet. combining our classifiers sequentially, we achieve 88.9% precision and 84.6% recall for discourse new entities.we expect our classifiers to provide a good prefiltering for coreference resolution systems, improving both their speed and performance.
learning non-isomorphic tree mappings for machine translation. often one may wish to learn a tree-to-tree mapping, training it on unaligned pairs of trees, or on a mixture of trees and strings. unlike previous statistical formalisms (limited to isomorphic trees), synchronous tsg allows local distortion of the tree topology. we reformulate it to permit dependency trees, and sketch em/viterbi algorithms for alignment, training, and decoding.
bilingual terminology acquisition from comparable corpora and phrasal translation to cross-language information retrieval. the present paper will seek to present an approach to bilingual lexicon extraction from non-aligned comparable corpora, phrasal translation as well as evaluations on cross-language information retrieval. a two-stages translation model is proposed for the acquisition of bilingual terminology from comparable corpora, disambiguation and selection of best translation alternatives according to their linguistics-based knowledge. different rescoring techniques are proposed and evaluated in order to select best phrasal translation alternatives. results demonstrate that the proposed translation model yields better translations and retrieval effectiveness could be achieved across japanese-english language pair.
discourse chunking: a tool in dialogue act tagging. discourse chunking is a simple way to segment dialogues according to how dialogue participants raise topics and negotiate them. this paper explains a method for arranging dialogues into chunks, and also shows how discourse chunking can be used to improve performance for a dialogue act tagger that uses a case-based reasoning approach.
finding non-local dependencies: beyond pattern matching. we describe an algorithm for recovering non-local dependencies in syntactic dependency structures. the pattern-matching approach proposed by johnson (2002) for a similar task for phrase structure trees is extended with machine learning techniques. the algorithm is essentially a classifier that predicts a non-local dependency given a connected fragment of a dependency structure and a set of structural features for this fragment. evaluating the algorithm on the penn treebank shows an improvement of both precision and recall, compared to the results presented in (johnson, 2002).
an ontology-based semantic tagger for ie system. in this paper, we present a method for the semantic tagging of word chunks extracted from a written transcription of conversations. this work is part of an ongoing project for an information extraction system in the field of maritime search and rescue (sar). our purpose is to automatically annotate parts of texts with concepts from a sar ontology. our approach combines two knowledge sources a sar ontology and the wordsmyth dictionary-thesaurus, and it uses a similarity measure for the classification. evaluation is carried out by comparing the output of the system with key answers of predefined extraction templates.
classifying recognition results for spoken dialog systems. this paper investigates the correlation between acoustic confidence scores as returned by speech recognizers with recognition quality. we report the results of two machine learning experiments that predict the word error rate of recognition hypotheses and the confidence error rate for individual words within them.
a computational treatment of korean temporal markers, oe and dongan. in this paper, we elucidate how korean temporal markers, oe and dongan contribute to specifying the event time and formalize it in terms of typed lambda calculus. we also present a computational method for constructing temporal representation of korean sentences on the basis of g grammar proposed by [renaud, 1992;1996].
dialog navigator : a spoken dialog q-a system based on large text knowledge base. this paper describes a spoken dialog q-a system as a substitution for call centers. the system is capable of making dialogs for both fixing speech recognition errors and for clarifying vague questions, based on only large text knowledge base. we introduce two measures to make dialogs for fixing recognition errors. an experimental evaluation shows the advantages of these measures.
the framenet data and software. the framenet project has developed a lexical knowledge base providing a unique level of detail as to the the possible syntactic realizations of the specific semantic roles evoked by each predicator, for roughly 7,000 lexical units, on the basis of annotating more than 100,000 example sentences extracted from corpora. an interim version of the framenet data was released in october, 2002 and is being widely used. a new, more portable version of the framenet software is also being made available to researchers elsewhere, including the spanish framenet project.this demo and poster will briefly explain the principles of frame semantics and demonstrate the new unified tools for lexicon building and annotation and also framesql, a search tool for finding patterns in annotated sentences. we will discuss the content and format of the data releases and how the software and data can be used by other nlp researchers.
an evaluation method of words tendency using decision tree. in every text, some words have frequency appearance and are considered as keywords because they have strong relationship with the subjects of their texts, these words frequencies change with time-series variation in a given period. however, in traditional text dealing methods and text search techniques, the importance of frequency change with time-series variation is not considered. therefore, traditional methods could not correctly determine index of word's popularity in a given period. in this paper, a new method is proposed to estimate automatically the stability classes (increasing, relatively constant, and decreasing) that indicate word's popularity with time-series variation based on the frequency change in past texts data. at first, learning data was produced by defining four attributes to measure frequency change of word quantitatively, these four attributes were extracted automatically from electronic texts.according to the comparison between the evaluation of the decision tree results and manually (human) results, f-measures of increasing, relatively constant and decreasing classes were 0.847, 0.851, and 0.768 respectively, and the effectiveness of this method is achieved.
spoken interactive odqa system: spiqa. we have been investigating an interactive approach for open-domain qa (odqa) and have constructed a spoken interactive odqa system, spiqa. the system derives disambiguating queries (dqs) that draw out additional information. to test the efficiency of additional information requested by the dqs, the system reconstructs the user's initial question by combining the addition information with question. the combination is then used for answer extraction. experimental results revealed the potential of the generated dqs.
a speech translation system with mobile wireless clients. we describe a new approach for elucidating the nonlinear degrees of freedom in a distribution of shapes depicted in digital images. by combining a deformation-based method for measuring distances between two shape configurations together with multidimensional scaling, a method for determining the number of degrees of freedom in a shape distribution is described. in addition, a method for visualizing the most representative modes of variation (underlying shape parameterization) in a nuclei shape distribution is also presented. the novel approach takes into account the nonlinear nature of shape manifolds and is related to the isomap algorithm. we apply the method to the task of analyzing the shape distribution of hela cell nuclei and conclude that approximately three parameters are responsible for their shape variation. excluding differences in size, translation, and orientation, these are: elongation, bending (concavity), and shifts in mass distribution. in addition, results show that, contrary to common intuition, the most likely nuclear shape configuration is not symmetric.
word sense disambiguation using pairwise alignment. in this paper, we proposed a new supervised word sense disambiguation (wsd) method based on a pairwise alignment technique, which is used generally to measure a similarity between dna sequences. the new method obtained 2.8%-14.2% improvements of the accuracy in our experiment for wsd.
on the applicability of global index grammars. we investigate global index grammars (gigs), a grammar formalism that uses a stack of indices associated with productions and has restricted context-sensitive power. we discuss some of the structural descriptions that gigs can generate compared with those generated by ligs. we show also how gigs can represent structural descriptions corresponding to hpsgs (pollard and sag, 1994) schemas.
deverbal compound noun analysis based on lexical conceptual structure. this paper proposes a principled approach for analysis of semantic relations between constituents in compound nouns based on lexical semantic structure. one of the difficulties of compound noun analysis is that the mechanisms governing the decision system of semantic relations and the representation method of semantic relations associated with lexical and contextual meaning are not obvious. the aim of our research is to clarify how lexical semantics contribute to the relations in compound nouns since such nouns are very productive and are supposed to be governed by systematic mechanisms. the results of applying our approach to the analysis of noun-deverbal compounds in japanese and english show that lexical conceptual structure contributes to the restrictional rules in compounds.
automatic collection of related terms from the web. this paper proposes a method of collecting a dozen terms that are closely related to a given seed term. the proposed method consists of three steps. the first step, compiling corpus step, collects texts that contain the given seed term by using search engines. the second step, automatic term recognition, extracts important terms from the corpus by using nakagawa's method. these extracted terms become the candidates for the final step. the final step, filtering step, removes inappropriate terms from the candidates based on search engine hits. an evaluation result shows that the precision of the method is 85%.
a novel approach to semantic indexing based on concept. this paper suggests the efficient indexing method based on a concept vector space that is capable of representing the semantic content of a document. the two information measure, namely the information quantity and the information ratio, are defined to represent the degree of the semantic importance within a document. the proposed method is expected to compensate the limitations of term frequency based methods by exploiting related lexical items. furthermore, with information ratio, this approach is independent of document length.
a limited-domain english to japanese medical speech translator built using regulus 2. we argue that verbal patient diagnosis is a promising application for limited-domain speech translation, and describe an architecture designed for this type of task which represents a compromise between principled linguistics-based processing on the one hand and efficient phrasal translation on the other. we propose to demonstrate a prototype system instantiating this architecture, which has been built on top of the open source regulus 2 platform. the prototype translates spoken yes-no questions about headache symptoms from english to japanese, using a vocabulary of about 200 words.
automatic error detection in the japanese learners' english spoken data. this paper describes a method of detecting grammatical and lexical errors made by japanese learners of english and other techniques that improve the accuracy of error detection with a limited amount of training data. in this paper, we demonstrate to what extent the proposed methods hold promise by conducting experiments using our learner corpus, which contains information on learners' errors.
automatic acquisition of named entity tagged corpus from world wide web. in this paper, we present a method that automatically constructs a named entity (ne) tagged corpus from the web to be used for learning of named entity recognition systems. we use an ne list and an web search engine to collect web documents which contain the ne instances. the documents are refined through sentence separation and text refinement procedures and ne instances are finally tagged with the appropriate ne categories. our experiments demonstrates that the suggested method can acquire enough ne tagged corpus equally useful to the manually tagged one without any human intervention.
a ranking model of proximal and structural text retrieval based on region algebra. this paper investigates an application of the ranked region algebra to information retrieval from large scale but unannotated documents. we automatically annotated documents with document structure and semantic tags by using taggers, and retrieve information by specifying structure represented by tags and words using ranked region algebra. we report in detail what kind of data can be retrieved in the experiments by this approach.
semantic classification of chinese unknown words. this paper describes a classifier that assigns semantic thesaurus categories to unknown chinese words (words not already in the cilin thesaurus and the chinese electronic dictionary, but in the sinica corpus). the focus of the paper differs in two ways from previous research in this particular area.prior research in chinese unknown words mostly focused on proper nouns (lee 1993, lee, lee and chen 1994, huang, hong and chen 1994, chen and chen 2000). this paper does not address proper nouns, focusing rather on common nouns, adjectives, and verbs. my analysis of the sinica corpus shows that contrary to expectation, most of unknown words in chinese are common nouns, adjectives, and verbs rather than proper nouns. other previous research has focused on features related to unknown word contexts (caraballo 1999; roark and charniak 1998). while context is clearly an important feature, this paper focuses on non-contextual features, which may play a key role for unknown words that occur only once and hence have limited context. the feature i focus on, following ciaramita (2002), is morphological similarity to words whose semantic category is known. my nearest neighbor approach to lexical acquisition computes the distance between an unknown word and examples from the cilin thesaurus based upon its morphological structure. the classifier improves on baseline semantic categorization performance for adjectives and verbs, but not for nouns.
a spoken dialogue interface for tv operations based on data collected by using woz method. the development of multi-channel digital broadcasting has generated a demand not only for new services but also for smart and highly functional capabilities in all broadcast-related devices. this is especially true of the television receivers on the viewer's side. with the aim of achieving a friendly interface that anybody can use with ease, we built a prototype interface system that operates a television through voice interactions using natural language. at the current stage of our research, we are using this system to investigate the usefulness and problem areas of the spoken dialogue interface for television operations.
integrating information extraction and automatic hyperlinking. this paper presents a novel information system integrating advanced information extraction technology and automatic hyper-linking. extracted entities are mapped into a domain ontology that relates concepts to a selection of hyperlinks. for information extraction, we use sprout, a generic platform for the development and use of multilingual text processing components. by combining finite-state and unification-based formalisms, the grammar formalism used in sprout offers both processing efficiency and a high degree of decalrativeness. the extralink demo system show-cases the extraction of relevant concepts from german texts in the tourism domain, offering the direct connection to associated web documents on demand.
totalrecall: a bilingual concordance for computer assisted translation and language learning. this paper describes a web-based english-chinese concordance system, total-recall, developed to promote translation reuse and encourage authentic and idiomatic use in second language writing. we exploited and structured existing high-quality translations from the bilingual sinorama magazine to build the concordance of authentic text and translation. novel approaches were taken to provide high-precision bilingual alignment on the sentence, phrase and word levels. a browser-based user interface (ui) is also developed for ease of access over the internet. users can search for word, phrase or expression in english or chinese. the web-based user interface facilitates the recording of the user actions to provide data for further research.