waashk/dblp huggingface dataset:
structure-based inference of xml similarity for fuzzy duplicate detection. fuzzy duplicate detection aims at identifying multiple representations of real-world objects stored in a data source, and is a task of critical practical relevance in data cleaning, data mining, or data integration. it has a long history for relational data stored in a single table (or in multiple tables with equal schema). algorithms for fuzzy duplicate detection in more complex structures, e.g., hierarchies of a data warehouse, xml data, or graph data have only recently emerged. these algorithms use similarity measures that consider the duplicate status of their direct neighbors, e.g., children in hierarchical data, to improve duplicate detection effectiveness. in this paper, we propose a novel method for fuzzy duplicate detection in hierarchical and semi-structured xml data. unlike previous approaches, it not only considers the duplicate status of children, but rather the probability of descendants being duplicates. probabilities are computed efficiently using a bayesian network. experiments show the proposed algorithm is able to maintain high precision and recall values, even when dealing with data containing a high amount of errors and missing information. our proposal is also able to outperform a state-of-the-art duplicate detection system on three different xml databases.
just in time indexing for up to the second search. e-commerce and intranet search systems require newly arriving content to be indexed and made available for search within minutes or hours of arrival. applications such as file system and email search demand even faster turnaround from search systems, requiring new content to become available for search almost instantaneously. however, incrementally updating inverted indices, which are the predominant datastructure used in search engines, is an expensive operation that most systems avoid performing at high rates. we present jiti, a just-in-time indexing component that allows searching over incoming content (nearly) as soon as that content reaches the system. jiti's main idea is to invest less in the preprocessing of arriving data, at the expense of a tolerable latency in query response time. it is designed for deployment in search systems that maintain a large main index and that rebuild smaller stop-press indices once or twice an hour. jiti augments such systems with instant retrieval capabilities over content arriving in between the stop-press builds. a main design point is for jiti to demand few computational resources, in particular ram and i/o. our experiments consisted of injecting several documents and queries per second concurrently into the system over half-hour long periods. we believe that there are search applications for which the combination of the workloads we experimented with and the response times we measured present a viable solution to a pressing problem.
ensembling bayesian network structure learning on limited data. in recent years, bagging method has been applied to learn bayesian networks (bns), especially on limited datasets. however, the bns learned using bagging method from limited datasets can be biased towards complex models. we present an efficient approach to produce more accurate bns from limited datasets. based on the markov condition of bn learning, we proposed a novel sampling method, called root nodes based sampling (rns), and a bns fusion method. the experimental results reveal that our ensemble method can achieve more accurate results in terms of accuracy on limited datasets.
expertise drift and query expansion in expert search. pseudo-relevance feedback, or query expansion, has been shown to improve retrieval performance in the adhoc retrieval task. in such a scenario, a few top-ranked documents are assumed to be relevant, and these are then used to expand and refine the initial user query, such that it retrieves a higher quality ranking of documents. however, there has been little work in applying query expansion in the expert search task. in this setting, query expansion is applied by assuming a few top-ranked candidates have relevant expertise, and using these to expand the query. nevertheless, retrieval is not improved as expected using such an approach. we show that the success of the application of query expansion is hindered by the presence of topic drift within the profiles of experts that the system considers. in this work, we demonstrate how topic drift occurs in the expert profiles, and moreover, we propose three measures to predict the amount of drift occurring in an expert's profile. finally, we suggest and evaluate ways of enhancing query expansion in expert search using our new insights. our results show that, once topic drift has been anticipated, query expansion can be successfully applied in a general manner in the expert search task.
dex: high-performance exploration on large graphs for information retrieval. link and graph analysis tools are important devices to boost the richness of information retrieval systems. internet and the existing social networking portals are just a couple of situations where the use of these tools would be beneficial and enriching for the users and the analysts. however, the need for integrating different data sources and, even more important, the need for high performance generic tools, is at odds with the continuously growing size and number of data repositories. in this paper we propose and evaluate dex, a high performance graph database querying system that allows for the integration of multiple data sources. dex makes graph querying possible in different flavors, including link analysis, social network analysis, pattern recognition and keyword search. the richness of dex shows up in the experiments that we carried out on the internet movie database (imdb). through a variety of these complex analytical queries, dex shows to be a generic and efficient tool on large graph databases.
automatic feature selection in the markov random field model for information retrieval. previous applications of the markov random field model for information retrieval have used manually chosen features. however, it is often difficult or impossible to know, a priori, the best set of features to use for a given task or data set. therefore, there is a need to develop automatic feature selection techniques. in this paper we describe a greedy procedure for automatically selecting features to use within the markov random field model for information retrieval. we also propose a novel, robust method for describing classes of textual information retrieval features. experimental results, evaluated on standard trec test collections, show that our feature selection algorithm produces models that are either significantly more effective than, or equally effective as, models with manually selected features, such as those used in the past.
wikify!: linking documents to encyclopedic knowledge. this paper introduces the use of wikipedia as a resource for automatic keyword extraction and word sense disambiguation, and shows how this online encyclopedia can be used to achieve state-of-the-art results on both these tasks. the paper also shows how the two methods can be combined into a system able to automatically enrich a text with links to encyclopedic knowledge. given an input document, the system identifies the important concepts in the text and automatically links these concepts to the corresponding wikipedia pages. evaluations of the system show that the automatic annotations are reliable and hardly distinguishable from manual annotations.
a knowledge-based search engine powered by wikipedia. this paper describes koru, a new search interface that offers effective domain-independent knowledge-based information retrieval. koru exhibits an understanding of the topics of both queries and documents. this allows it to (a) expand queries automatically and (b) help guide the user as they evolve their queries interactively. its understanding is mined from the vast investment of manual effort and judgment that is wikipedia. we show how this open, constantly evolving encyclopedia can yield inexpensive knowledge structures that are specifically tailored to expose the topics, terminology and semantics of individual document collections. we conducted a detailed user study with 12 participants and 10 topics from the 2005 trec hard track, and found that koru and its underlying knowledge base offers significant advantages over traditional keyword search. it was capable of lending assistance to almost every query issued to it; making their entry more efficient, improving the relevance of the documents they return, and narrowing the gap between expert and novice seekers.
hybrid results merging. the problem of results merging in distributed information retrieval environments has been approached by two different directions in research. estimation approaches attempt to calculate the relevance of the returned documents through ad-hoc methodologies (weighted score merging, regression etc) while download approaches, download all the documents locally, partially or completely, in order to estimate "first hand" their relevance. both have their advantages and disadvantages. it is assumed that download algorithms are more effective but they are very expensive in terms of time and bandwidth. estimation approaches on the other hand, usually rely on document relevance scores being returned by the remote collections in order to achieve maximum performance. in addition to that, regression algorithms, which have proved to be more effective than weighted scores merging, rely on a significant number of overlap documents in order to function effectively, practically requiring multiple interactions with the remote collections. the new algorithm that is introduced reconciles the above two approaches, combining their strengths, while minimizing their weaknesses. it is based on downloading a limited, selected number of documents from the remote collections and estimating the relevance of the rest through regression methodologies. the proposed algorithm is tested in a variety of settings and its performance is found to be better than estimation approaches, while approximating that of download.
automatic call section segmentation for contact-center calls. this paper presents a svm (support vector machine) classification system which divides contact-center call transcripts into "greeting", "question", "refine", "research", "resolution", "closing" and "out-of-topic" sections. this call section segmentation is useful to improve search and retrieval functions and to provide more detailed statistics on calls. we use an off-the-shelf automatic speech recognition (asr) system to generate call transcripts from recorded calls between customers and service representatives. we first classify an individual utterance into a call section by applying the svm classifier and then merge adjacent utterances classified into a same call section. we experiment with the proposed system on 100 automatically transcribed calls. the 10-fold cross validation shows 87.2% classification accuracy. we also compare the proposed algorithm with two other approaches - the most frequent section only method and a maximum entropy-based segmentation. the evaluation shows that our system's accuracy is 12% higher than the first baseline system and 6% higher than the second baseline system respectively.
grid-based subspace clustering over data streams. a real-life data stream usually contains many dimensions and some dimensional values of its data elements may be missing. in order to effectively extract the on-going change of a data stream with respect to all the subsets of the dimensions of the data stream, a grid-based subspace clustering algorithm is proposed in this paper. given an n-dimensional data stream, the on-going distribution statistics of data elements in each one-dimension data space is firstly monitored by a list of grid-cells called a sibling list. once a dense grid-cell of a first-level sibling list becomes a dense unit grid-cell, new second-level sibling lists are created as its child nodes in order to trace any cluster in all possible two-dimensional rectangular subspaces. in such a way, a sibling tree grows up to the nth level at most and a k-dimensional subcluster can be found in the kth level of the sibling tree. the proposed method is comparatively analyzed by a series of experiments to identify its various characteristics.
mining web multi-resolution community-based popularity for information retrieval. the pagerank algorithm is used in web information retrieval to calculate a single list of popularity scores for each page in the web. these popularity scores are used to rank query results when presented to the user. by using the structure of the entire web to calculate one score per document, we are calculating a general popularity score, not particular to any community. therefore, the pagerank scores are more suited to general queries. in this paper, we introduce a more general form of pagerank, using web multi-resolution community-based popularity scores, where each document obtains a popularity score dependent on a given web community. when a query is related to a specific community, we choose the associated set of popularity scores and order the query results accordingly. using web-community based popularity scores, we achieved an 11% increase in precision over pagerank.
lightweight web-based fact repositories for textual question answering. since answers to fact-seeking questions usually reside within small factual text nuggets, often "hidden" within full-length documents, their relevance to a question is not necessarily correlated to the relevance of the full-length document to the question. yet previous approaches to open-domain textual question answering from large document collections quasi-unanimously employ a document retrieval stage, in order to apply widely different, often expensive answer mining techniques to only a small subset of documents. depending on the collection size, 95% or more of the documents in the collection (much more in the case of the web) are left out of the selected subset for any given query, and thus become invisible to subsequent processing stages for actual answer mining. this paper introduces a new model for answer retrieval for question answering. the collection is distilled offline into large repositories of facts. each fact constitutes a potential direct answer to questions seeking a particular kind of entity or relation, such as questions asking about the date of particular events. question answering becomes equivalent to online fact retrieval, which greatly simplifies the de-facto system architecture for fact-seeking question answering. in addition to simplicity, experiments on a fact repository acquired from approximately a billion web documents illustrate the impact of fact repositories in extracting accurate answers to a standard evaluation set of open-domain test questions and additional sets of domain-specific questions.
weakly-supervised discovery of named entities using web search queries. a seed-based framework for textual information extraction allows for weakly supervised extraction of named entities from anonymized web search queries. the extraction is guided by a small set of seed named entities, without any need for handcrafted extraction patterns or domain-specific knowledge, allowing for the acquisition of named entities pertaining to various classes of interest to web search users. inherently noisy search queries are shown to be a highly valuable, albeit little explored, resource for web-based named entity discovery.
the role of documents vs. queries in extracting class attributes from text. challenging the implicit reliance on document collections, this paper discusses the pros and cons of using query logs rather than document collections, as self-contained sources of data in textual information extraction. the differences are quantified as part of a large-scale study on extracting prominent attributes or quantifiable properties of classes (e.g., top speed, price and fuel consumption for carmodel) from unstructured text. in a head-to-head qualitative comparison, a lightweight extraction method produces class attributes that are 45% more accurate on average, when acquired from query logs rather than web documents.
using word similarity to eradicate junk emails. emails are one of the most commonly used modern communication media these days; however, unsolicited emails obstruct this otherwise fast and convenient technology for information exchange and jeopardize the continuity of this popular communication tool. waste of valuable resources and time and exposure to offensive content are only a few of the problems that arise as a result of junk emails. in addition, the monetary cost of processing junk emails reaches billions of dollars per year and is absorbed by public users and internet service providers. even though there has been extensive work in the past dedicated to eradicate junk emails, none of the existing junk email detection approaches has been highly successful in solving these problems, since spammers have been able to infiltrate existing detection techniques. in this paper, we present a new tool, junex, which relies on the content similarity of emails to eradicate junk emails. junex compares each incoming email to a core of emails marked as junk by each individual user to identify unwanted emails while reducing the number of legitimate emails treated as junk, which is critical. conducted experiments on junex verify its high accuracy.
proximity-based document representation for named entity retrieval. one aspect in which retrieving named entities is different from retrieving documents is that the items to be retrieved - persons, locations, organizations - are only indirectly described by documents throughout the collection. much work has been dedicated to finding references to named entities, in particular to the problems of named entity extraction and disambiguation. however, just as important for retrieval performance is how these snippets of text are combined to build named entity representations. we focus on the trec expert search task where the goal is to identify people who are knowledgeable on a specific topic. existing language modeling techniques for expert finding assume that terms and person entities are conditionally independent given a document. we present theoretical and experimental evidence that this simplifying assumption ignores information on how named entities relate to document content. to address this issue, we propose a new document representation which emphasizes text in proximity to entities and thus incorporates sequential information implicit in text. our experiments demonstrate that the proposed model significantly improves retrieval performance. the main contribution of this work is an effective formal method for explicitly modeling the dependency between the named entities and terms which appear in a document.
latent semantic fusion model for image retrieval and annotation. this paper studies the effect of latent semantic analysis (lsa) on two different tasks: multimedia document retrieval (mdr) and automatic image annotation (aia). the contributions of this paper are twofold. first, to the best of our knowledge, this work is the first study of the influence of lsa on the retrieval of a significant number of multimedia documents (i.e. collection of 20000 tourist images). second, it shows how different image representations (region-based and keypoint-based) can be combined by lsa to improve automatic image annotation. the document collections used for these experiments are the corel photo collection and imageclef 2006 collection.
modeling historical and future movements of spatio-temporal objects in moving objects databases. spatio-temporal databases deal with geometries changing over time. in general, geometries do not only change discretely but continuously; hence we are dealing with moving objects. in the past, a few moving object data models and query languages have been proposed. each of them supports either historical movements or future movements but not both together. consequently, queries that start in the past and extend into the future cannot be supported. to model both historical and future movements of an object, two separate concepts with different properties are required, and extra attention is necessary to avoid their conflicts. furthermore, current definitions of moving objects are too general and vague. it is unclear how a moving object is allowed to move through space and time. for instance, the continuity or discontinuity of motion is not specified. in this paper, we propose a new moving object data model called balloon model which provides integrated support for both historical and future movements of moving objects. as part of the model, we provide formal definitions of moving objects with respect to their past and future movements. all kinds of queries including past queries, future queries, and queries that start in the past and end in the future are supported in our model.
semantic components enhance retrieval of domain-specific documents. we seek to leverage knowledge about information organization in a domain to effectively and efficiently meet targeted information needs of expert users. the semantic components model represents document content in a manner that is complementary to full text and keyword indexing. semantic component instances are segments of text about a particular aspect of the main topic of the document and may not correspond to structural elements in the document. this paper describes the semantic components model and presents experimental evidence from a large interactive searching study showing that semantic components, used to supplement full text and keyword indexing and to extend the query language, enhanced the retrieval of domain-specific documents in response to realistic queries posed by real users.
satisfaction balanced mediation. we consider a distributed information system that allows autonomous consumers to query autonomous providers. we focus on the problem of query allocation from a new point of view, by considering consumers and providers' satisfaction in addition to query load. we define satisfaction as a long-run notion based on the consumers and providers' preferences. we propose and validate a mediation process, called sbmediation, which is compared to capacity based query allocation. the experimental results show that sbmediation significantly outperforms capacity based when confronted to autonomous participants.
efficient web matrix processing based on dual reordering. pagerank is one of the most important ranking techniques used in search engines nowadays. since billions of pages already existed in web, many pagerank acceleration techniques were explored by many researchers. however, in the real web, just like the existence of pages without out-links, there are many pages without in-links. (in this paper, a web page without in-links is called a reverse-dangling page.) given the state of the art, we propose a new reordered pagerank algorithm, called two-way reordered pagerank algorithm, which exploits both dangling nodes and reverse-dangling nodes to reduce the computational complexity of the pagerank vector.
semantic verification in an online fact seeking environment. many artificial intelligence tasks, such as automated question answering, reasoning or heterogeneous database integration, involve verification of a semantic category (e.g. "coffee" is a drink, "red" is a color, while "steak" is not a drink and "big" is not a color). we present a novel algorithm to automatically validate a semantic category. contrary to the methods suggested earlier, our approach does not rely on any manually codified knowledge but instead capitalizes on the diversity of topics and word usage on the world wide web. we have tested our approach within our online fact-seeking (question answering) environment. when tested on the trec questions that expect the answer to belong to a specific semantic category, our approach has improved the accuracy by up to 14% depending on the model and metrics used.