waashk/dblp huggingface dataset:
a spam resistant family of concavo-convex ranks for link analysis. a parameterized family of non-linear, link analytic ranking functions is proposed that includes pagerank as a special case and uses the convexity property of those functions to be more resistant to link spam attacks. a contribution of the paper is the construction of such a scheme with provable uniqueness and convergence guarantees. the paper also demonstrates that even in an unlabelled scenario this family can have spam resistance comparable to trustrank [3] that uses labels of spam or nat-spam on a training set. the proposed method can use labels, if available, to improve its performance to provide state of the art level of link spam protection.
incorporating place name extents into geo-ir ranking. this paper proposes a novel geo-ir ranking method that realizes effective searches that emphasize the user's immediate surroundings. it assesses the extent implied by place names in documents and then emphasizes place names that are highly specific in terms of identifying locations.
ce2: towards a large scale hybrid search engine with integrated ranking support. the web contains a large amount of documents and increasingly, also semantic data in the form of rdf triples. many of these triples are annotations that are associated with documents. while structured query is the principal mean to retrieve semantic data, keyword queries are typically used for document retrieval. clearly, a form of hybrid search that seamlessly integrates these formalisms to query both documents and semantic data can address more complex information needs. in this paper, we present ce2, an integrated solution that leverages mature database and information retrieval technologies to tackle challenges in hybrid search on the large scale. for scalable storage, ce2 integrates database with inverted indices. hybrid query processing is supported in ce2 through novel algorithms and data structures, which allow for advanced ranking schemes to be integrated more tightly into the process. experiments conducted on dbpedia and wikipedia show that ce2 can provide good performance in terms of both effectiveness and efficiency.
an empirical study of required dimensionality for large-scale latent semantic indexing applications. the technique of latent semantic indexing is used in a wide variety of commercial applications. in these applications, the processing time and ram required for svd computation, and the processing time and ram required during lsi retrieval operations are all roughly linear in the number of dimensions, k, chosen for the lsi representation space. in large-scale commercial lsi applications, reducing k values could be of significant value in reducing server costs. this paper explores the effects of varying dimensionality. the approach taken here focuses on term comparisons. pairs of terms are considered which have strong real-world associations. the proximities of members of these pairs in the lsi space are compared at multiple values of k. the testing is carried out for collections of from one to five million documents. for the five million document collection, a value of k ≈ 400 provides the best performance. the results suggest that there is something of an 'island of stability' in the k = 300 to 500 range. the results also indicate that there is relatively little room to employ k values outside of this range without incurring significant distortions in at least some term-term correlations.
a matrix-based approach for semi-supervised document co-clustering. in order to derive high quality information from text, the field of text mining has advanced swiftly from simple document clustering to co-clustering documents and words. however, document co-clustering without any prior knowledge or background information is a challenging problem. in this paper, we propose a semi-supervised non-negative matrix factorization (ss-nmf) based framework for document co-clustering. our method computes a new word-document matrix by incorporating user provided constraints through distance metric learning. using an iterative algorithm, we perform tri-factorization of the new matrix to infer the document and word clusters. through extensive experiments conducted on publicly available data sets, we demonstrate the superior performance of ss-nmf for document co-clustering.
simultaneous multilingual search for translingual information retrieval. we consider the problem of translingual information retrieval, where monolingual searchers issue queries in a different language than the document language(s) and the results must be returned in the language they know, the query language. we present a framework for translingual ir that integrates document translation and query translation into the retrieval model. the corpus is represented as an aligned, jointly indexed "pseudo-parallel" corpus, where each document contains the text of the document along with its translation into the query language. the queries are formulated as multilingual structured queries, where each query term and its translations into the document language(s) are treated as synonym sets. this model leverages simultaneous search in multiple languages against jointly indexed documents to improve the accuracy of results over search using document translation or query translation alone. for query translation, we compared a statistical machine translation (smt) approach to a dictionary-based approach. we found that using a wikipedia-derived dictionary for named entities combined with an smt-based dictionary worked better than smt alone. simultaneous multilingual search also has other important features suited to translingual search, since it can provide an indication of poor document translation when a match with the source document is found. we show how close integration of clir and smt allows us to improve result translation in addition to ir results.
web-scale named entity recognition. automatic recognition of named entities such as people, places, organizations, books, and movies across the entire web presents a number of challenges, both of scale and scope. data for training general named entity recognizers is difficult to come by, and efficient machine learning methods are required once we have found hundreds of millions of labeled observations. we present an implemented system that addresses these issues, including a method for automatically generating training data, and a multi-class online classification training method that learns to recognize not only high level categories such as place and person, but also more fine-grained categories such as soccer players, birds, and universities. the resulting system gives precision and recall performance comparable to that obtained for more limited entity types in much more structured domains such as company recognition in newswire, even though web documents often lack consistent capitalization and grammatical sentence construction.
unsolved problems in search: (and how we approach them). search applications have become ubiquitous and very successful. major advances have been made in understanding how to deliver effective results very efficiently for a class of queries. as the range of applications broaden to include web search, desktop search, enterprise search, vertical search, social search, etc., the number of new research challenges has appeared to grow rather than shrink. many of these challenges are variations on underlying themes and principles that information retrieval has focused on for more than 40 years. in this talk, several unsolved problems arising from new search applications will be discussed and some potential paths to solutions for these problems will be outlined.
a note on search based forecasting of ad volume in contextual advertising. in contextual advertising, estimating the number of impressions of an ad is critical in planning and budgeting advertising campaigns. however, producing this forecast, even within large margins of error, is quite challenging. we attack this problem by simulating the presence of a given ad with its associated bid over historical data, involving billions of impressions. this apparently enormous computational task is reduced to a search task involving only the set of distinct pages in the data. furthermore the search is made more efficient using a two-level search process. experimental results show that our approach can accurately forecast the expected number of impressions of contextual ads in real time.
social tags: meaning and suggestions. this paper aims to quantify two common assumptions about social tagging: (1) that tags are "meaningful" and (2) that the tagging process is influenced by tag suggestions. for (1), we analyze the semantic properties of tags and the relationship between the tags and the content of the tagged page. our analysis is based on a corpus of search keywords, contents, titles, and tags applied to several thousand popular web pages. among other results, we find that the more popular tags of a page tend to be the more meaningful ones. for (2), we develop a model of how the influence of tag suggestions can be measured. from a user study with over 4,000 participants, we conclude that roughly one third of the tag applications may be induced by the suggestions. our results would be of interest for designers of social tagging systems and are a step towards understanding how to best leverage social tags for applications such as search and information extraction.
kernel methods, syntax and semantics for relational text categorization. previous work on natural language processing for information retrieval has shown the inadequateness of semantic and syntactic structures for both document retrieval and categorization. the main reason is the high reliability and effectiveness of language models, which are sufficient to accurately solve such retrieval tasks. however, when the latter involve the computation of relational semantics between text fragments simple statistical models may result ineffective. in this paper, we show that syntactic and semantic structures can be used to greatly improve complex categorization tasks such as determining if an answer correctly responds to a question. given the high complexity of representing semantic/syntactic structures in learning algorithms, we applied kernel methods along with support vector machines to better exploit the needed relational information. our experiments on answer classification on web and trec data show that our models greatly improve on bag-of-words.
a light weighted damage tracking quarantine and recovery scheme for mission-critical database systems. as online applications gain popularity in today's e-business world, surviving dbms from an attack is becoming crucial because of the increasingly critical role that database servers are playing. although a number of research projects have been done to tackle the emerging data corruption threats, existing mechanisms are still limited in meeting four highly desired requirements: near zero run time overhead, zero system down time. in this paper, we propose trace, a light weighted database damage tracking, quarantine, and recovery (dtqr) solution with negligible run time overhead.
minimum-effort driven dynamic faceted search in structured databases. in this paper, we propose minimum-effort driven navigational techniques for enterprise database systems based on the faceted search paradigm. our proposed techniques dynamically suggest facets for drilling down into the database such that the cost of navigation is minimized. at every step, the system asks the user a question or a set of questions on different facets and depending on the user response, dynamically fetches the next most promising set of facets, and the process repeats. facets are selected based on their ability to rapidly drill down to the most promising tuples, as well as on the ability of the user to provide desired values for them. our facet selection algorithms also work in conjunction with any ranked retrieval model where a ranking function imposes a bias over the user preferences for the selected tuples. our methods are principled as well as efficient, and our experimental study validates their effectiveness on several application scenarios.
can phrase indexing help to process non-phrase queries? modern web search engines, while indexing billions of web pages, are expected to process queries and return results in a very short time. many approaches have been proposed for efficiently computing top-k query results, but most of them ignore one key factor in the ranking functions of commercial search engines - term-proximity, which is the metric of the distance between query terms in a document. when term-proximity is included in ranking functions, most of the existing top-k algorithms will become inefficient. to address this problem, in this paper we propose to build a compact phrase index to speed up the search process when incorporating the term-proximity factor. the compact phrase index can help more accurately estimate the score upper bounds of unknown documents. the size of the phrase index is controlled by including a small portion of phrases which are possibly helpful for improving search performance. phrase index has been used to process phrase queries in existing work. it is, however, to the best of our knowledge, the first time that phrase index is used to improve the performance of generic queries. experimental results show that, compared with the state-of-the-art top-k computation approaches, our approach can reduce average query processing time to 1/5 for typical setttings.
medsearch: a specialized search engine for medical information retrieval. people are thirsty for medical information. existing web search engines often cannot handle medical search well because they do not consider its special requirements. often a medical information searcher is uncertain about his exact questions and unfamiliar with medical terminology. therefore, he sometimes prefers to pose long queries, describing his symptoms and situation in plain english, and receive comprehensive, relevant information from search results. this paper presents medsearch, a specialized medical web search engine, to address these challenges. medsearch uses several key techniques to improve its usability and the quality of search results. first, it accepts queries of extended length and reforms long queries into shorter queries by extracting a subset of important and representative words. this not only significantly increases the query processing speed but also improves the quality of search results. second, it provides diversified search results. lastly, it suggests related medical phrases to help the user quickly digest search results and refine the query. we evaluated medsearch using medical questions posted on medical discussion forums. the results show that medsearch can handle various medical queries effectively and efficiently.
efficient techniques for document sanitization. sanitization of a document involves removing sensitive information from the document, so that it may be distributed to a broader audience. such sanitization is needed while declassifying documents involving sensitive or confidential information such as corporate emails, intelligence reports, medical records, etc. in this paper, we present the erase framework for performing document sanitization in an automated manner. erase can be used to sanitize a document dynamically, so that different users get different views of the same document based on what they are authorized to know. we formalize the problem and present algorithms used in erase for finding the appropriate terms to remove from the document. our preliminary experimental study demonstrates the efficiency and efficacy of the proposed algorithms.
structural relevance: a common basis for the evaluation of structured document retrieval. this paper presents a unified framework for the evaluation of a range of structured document retrieval (sdr) approaches and tasks. the framework is based on a model of tree retrieval, evaluated using a novel extension of the structural elevance (sr) measure. the measure replaces the assumption of independence in traditional information retrieval (ir) with a notion of redundancy that takes into account the user navigation inside documents while seeking relevant information. unlike existing metrics for sdr, our proposed framework does not require the computation of an ideal ranking which has, thus far, prevented the practical application of such measures. instead, sr builds on a markovian model of user navigation that can be estimated through the use of structural summaries. the results of this paper (supported by experimental validation using inex data) show that sr defined over a tree retrieval model can provide a common basis for the evaluation of sdr approaches across various structured search tasks.
tag-based filtering for personalized bookmark recommendations. this paper investigates using social tags for the purpose of making personalized content recommendations. our tag-based recommender creates a personalized bookmark recommendation model for each user based on "current" and "general interest" tags, defined by different time intervals.
a pattern-based analyzer for french in the context of spoken language translation: first prototype and evaluation. in this paper, we describe a first prototype of a pattern-based analyzer developed in the context of a speech-to-speech translation project using a pivot-based approach (the pivot is called if). the chosen situation involves a french client talking to an italian travel agent (both in their own language) to organize a stay in the trentino area.an if consists of a dialogue act, and a list, possibly empty, of argument values. the analyzer applies a "phrase spotting" mechanism on the output of the speech recognition module. it finds well-formed phrases corresponding to argument values. a dialogue act is then built according to the instantiated arguments and some other features of the input.the current version of the prototype has been involved in an evaluation campaign on an unseen corpus of four dialogues consisting of 235 speech turns. the results are given and commented in the last part of the paper. we think they pave the way for future enhancements to both the coverage and the development methodology.
lhip: extended dcgs for configurable robust parsing. we present lhip, a system for incremental grammar development using an extended dcg for malism. the system uses a robust island-based parsing method controlled by user-defined performance thresholds.
uniform recognition for acyclic context-sensitive grammars is np-complete. context-sensitive grammars in which each rule is of the form &alpha;z&beta; &rarr; &alpha;&gamma;&beta; are acyclic if the associated context-free grammar with the rules z &rarr; &gamma; is acyclic. the problem whether an input string is in the language generated by an acyclic context-sensitive grammar is np-complete.
feasible learnability of formal grammars and the theory of natural language acquisition. we propose to apply a complexity theoretic notion of feasible learnability called "polynomial learnability" to the evaluation of grammatical formalisms for linguistic description. polynomial learnability was originally defined by valiant in the context of boolean concept learning and subsequently generalized by blumer et al. to infinitary domains. we give a clear, intuitive exposition of this notion of learnability and what characteristics of a collection of languages may or many not help feasible learn ability under this paradigm. in particular, we present a novel, nontrivial constraint on the degree of "locality" of grammars which allows a rich class of mildly context sensitive languages to be feasibly learnable. we discuss possible implications of this observation to the theory of natural language acquisition.
improved iterative scaling can yield multiple globally optimal models with radically differing performance levels. log-linear models can be efficiently estimated using algorithms such as improved iterative scaling (iis) (lafferty et al., 1997). under certain conditions and for a particular class of problems, iis is guaranteed to approach both the maximum-likelihood and maximum entropy solution. this solution, in likelihood space, is unique. unfortunately, in realistic situations, multiple solutions may exist, all of which are equivalent to each other in terms of likelihood, but radically different from each other in terms of performance. we show that this behaviour can occur when a model contains overlapping features and the training material is sparse. experimental results, from the domain of parse selection for stochastic attribute value grammars, shows the wide variation in performance that can be found when estimating models using iis. further results show that the influence of the initial model can be diminished by selecting either uniform weights, or else by model averaging.
a kana-kanji translation system for non-segmented input sentences based on syntactic and semantic analysis. this paper presents a disambiguation approach for translating non-segmented-kana into kanji. the method consists of two steps. in the first step, an input sentence is analyzed morphologically and ambiguous morphemes are stored in a network form. in the second step, the best path, which is a string of morphemes, is selected by syntactic and semantic analysis based on case grammar. in order to avoid the combinatorial explosion of possible paths, the following heuristic search method is adopted. first, a path that contains the smallest number of weighted-morphemes is chosen as the quasi-best path by a best-first-search technique. next, the restricted range of morphemes near the quasi-best path is extracted from the morpheme network to construct preferential paths.an experimental system incorporating large dictionaries has been developed and evaluated. a translation accracy of 90.5% was obtained. this can be improved to about 95% by optimizing the dictionaries.
bootstrapping bilingual data using consensus translation for a multilingual instant messaging system. one of the primary issues in training statistical translation models is the paucity of bilingual data. in this paper, we propose techniques to alleviate the bilingual data bottleneck by creating a consensus from translations of monolingual data provided by several off-the-shelf translation engines. we compute the consensus alignment using a multi-sequence alignment algorithm used for dna sequence alignment. we present an application of this technique to bootstrap bilingual data for the general domain of instant messaging. we train hierarchical statistical translation models on the bootstrapped bilingual data and show that the resulting statistical translation model outperforms each individual off-the-shelf translation system.
a learning of object structures by verbalism. in this paper an attempt of learning by verbalism is shown in order to create the models for ancidentification of unknown objects. when we expect a computer to recognize objects, the models of them must be given to it, however there are cases where some objects may not be, matched to the models or there is no model with which object is compared. at that time, this system can augment or create new descriptions by being explicitly taught using verbal instructions.
exploiting a probabilistic hierarchical model for generation. previous stochastic approaches to generation do not include a tree-based representation of syntax. while this may be adequate or even advantageous for some applications, other applications profit from using as much syntactic knowledge as is available, leaving to a stochastic model only those issues that are not determined by the grammar. we present initial results showing that a tree-based model derived from a tree-annotated corpus improves on a tree model derived from an unannotated corpus, and that a tree-based stochastic model with a hand-crafted grammar outperforms both.