waashk/dblp huggingface dataset:
y!q: contextual search at the point of inspiration. contextual search tries to better capture a user's information need by augmenting the user's query with contextual information extracted from the search context (for example, terms from the web page the user is currently reading or a file the user is currently editing).this paper presents y!q---a first of its kind large-scale contextual search system---and provides an overview of its system design and architecture. y!q solves two major problems. first, how to capture high quality search context. second, how to use that context in a way to improve the relevancy of search queries. to address the first problem, y!q introduces an information widget that captures precise search context and provides convenient access to its functionality at the point of inspiration. for example, y!q can be easily embedded into web pages using a web api, or it can be integrated into a web browser toolbar. this paper provides an overview of y!q's user interaction design, highlighting its novel aspects for capturing high quality search context.to address the second problem, y!q uses a semantic network for analyzing search context, possibly resolving ambiguous terms, and generating a contextual digest comprising its key concepts. this digest is passed through a query planner and rewriting framework for augmenting a user's search query with relevant context terms to improve the overall search relevancy and experience. we show experimental results comparing contextual y!q search results side-by-side with regular yahoo! web search results. this evaluation suggests that y!q results are considered significantly more relevant.the paper also identifies interesting research problems and argues that contextual search may represent the next major step in the evolution of web search engines.
efficient and effective server-sided distributed clustering. clustering has become an increasingly important task in modern application domains where the data are originally located at different sites. in order to create a central clustering, all clients have to transmit their data to a central server. due to technical limitations and security aspects, at the central site often only vague object descriptions are available. the server then has to carry out the clustering based on vague and uncertain data. in a recent paper, an approach for clustering uncertain data was proposed based on the concept of medoid clusterings. the idea of this approach is to create first several sample clusterings. then based on suitable distance functions between clusterings the most average clustering, i.e. the medoid clustering, was determined. in this paper, we extend this approach for partitioning clustering algorithms and propose to compute a centroid clustering based on these input sample clusterings. these centroid clusterings are new artificial clusterings which minimize the distance to all the sample clusterings.
towards smarter documents. document analysis research typically focuses on document image understanding or classic problems in text classification, clustering, summarization and discovery. while that is an important aspect of document management, in practice, documents lifecycles are often determined by the context of the business process that they are relevant to. it therefore becomes necessary for the document analysis techniques to recognize and leverage the contextual information provided by a supporting schema and business process. this paper presents an intelligent document management framework with relevant document analysis, metadata extraction, and business process association algorithms and methodology. the architecture supporting this framework seamlessly integrates a runtime environment with an authoring environment by combining relational data modeling tools with document classification techniques. the runtime environment accepts incoming documents, classifies the document, extracts metadata and executes customized business logic. the authoring environment supports the association of a class of documents with a relational document schema, identification of attribute values that must be extracted automatically, generation of relevant business logic, and deployment of authoring artifacts into the runtime architecture. we demonstrate the use of this framework with representative real-world document transformative applications.
a clustering algorithm for asymmetrically related data with applications to text mining. clustering techniques find a collection of subsets of a data set such that the collection satisfies a criterion that is dependent on a relation defined on the data set. the underlying relation is traditionally assumed to be symmetric. however, there exist many practical scenarios where the underlying relation is asymmetric. one example of an asymmetric relation in text analysis is the inclusion relation, i.e., the inclusion of the meaning of a block of text in the meaning of another block. in this paper, we consider the general problem of clustering of asymmetrically related data and propose an algorithm to cluster such data. to demonstrate its usefulness, we consider two applications in text mining: (1) summarization of short documents, and (2) generation of a concept hierarchy from a set of documents. our experiments show that the performance of the proposed algorithm is superior to that of more traditional algorithms.
biasing web search results for topic familiarity. depending on a web searcher's familiarity with a query's target topic, it may be more appropriate to show her introductory or advanced documents. the trec hard [1] track defined topic familiarity as meta-data associated with a user's query. we instead define a user-independent and query-independent model of topic-familiarity required to read a document, so it can be matched to a given user in response to a query. an introductory web page is defined as a web page that doesn't presuppose any background knowledge of the topic it is on, and to an extent introduces or defines the key terms in the topic. while an advanced web page is defined as a web page that assumes sufficient background knowledge of the topic it is on, and familiarity with the key technical/ important terms in the topic, and potentially builds on them. we develop a method for biasing the initial mix of documents returned by a search engine to increase the number of documents of desired familiarity level up to position 5, and up to position 10. our method involves building a supervised text classifier, incorporating features based on reading level, the distribution of stop-words in the text, and non-text features such as average line-length. using this familiarity classifier, we achieve statistically significant improvements at reranking the result set to show introductory documents higher up the ranked list. our classifier can be seamlessly integrated into current search engine technology without involving any major modifications to existing architectures.
query expansion using domain adapted, weighted thesaurus in an extended boolean model. in this paper, we address there important issues with query expansion using a thesaurus; how to give weights to the terms in expanded queries, how to select additional search terms in the thesaurus, and how to enrich the terms in the manual thesaurus (namely, thesaurus reconstruction). to weight the terms in expanded queries, we construct the weighted thesaurus that has a similarity value between the terms in the thesaurus, using statistical co-occurrence in a corpus. to enrich the terms in the manual thesaurus, domain dependent terms which occur in a corpus are inserted into the weighted thesaurus using the co-occurrence information. in this paper, the reconstructed thesaurus with weights is defined as a domain-adapted, weighted thesaurus. then we explain query expansion using the domain-adapted, weighted thesaurus in an extended boolean retrieval model. to select additional search terms during query expansion, our model uses semi-automatic query expansion and a restriction method. in the experiments, our system had almost twice the recall of the boolean retrieval system not using the thesaurus or the query expansion retrieval system using the original thesaurus. and also, the precision of our system was almost the same precision as the other systems.
yahoo! as an ontology: using yahoo! categories to describe documents. we suggest that one (or a collection) of names of yahoo! (or any other www indexer's) categories can be used to describe the content of a document. such categories offer a standardized and universal way for referring to or describing the nature of real world objects, activities, documents and so on, and may be used (we suggest) to semantically characterize the content of documents. www indices, like yahoo! provide a huge hierarchy of categories (topics) that touch every aspect of human endeavors. such topics can be used as descriptors, similarly to the way librarians use for example, the library of congress cataloging system to annotate and categorize books.in the course of investigating this idea, we address the problem of automatic categorization of webpages in the yahoo! directory. we use telltale as our classifier; telltale uses n-grams to compute the similarity between documents. we experiment with various types of descriptions for the yahoo! categories and the webpages to be categorized. our findings suggest that the best results occur when using the very brief descriptions of the yahoo! categorized entries; these brief descriptions are provided either by the entries' submitters or by the yahoo! human indexers and accompany most yahoo!-indexed entries.
automatic identification of best entry points for focused structured document retrieval. focussed structured document retrieval employs the concept of best entry points (beps), which are intended to provide optimal starting-points from which users can browse to relevant document components. this paper describes two small-scale studies, using experimental data from the shakespeare user study, which developed and evaluated different approaches to the problem of automatic identification of beps.
efficient synchronization for mobile xml data. many handheld applications receive data from a primary database server and operate in an intermittently connected environment these days. they maintain data consistency with data sources through sychronization. in certain applications such as sales force automation, it is highly desirable if updates on the data source can be reflected at the handheld applications immediately. this paper proposes an efficient method to synchronize xml data on multiple mobile devices. each device retrieves and caches a local copy of data from the database source based on a regular path expression. these local copies may be overlapping or disjoint with each other. an efficient mechanism is proposed to find all the disjoint copies to avoid unnecessary synchronizations. each update to the data source will then be checked to identify all handheld applications which are affected by the update. communication costs can be further reduced by eliminating the forwarding of unnecessary operations to groups of mobile clients.
application of a symbolico-connectionist approach for the design of a highly interactive documentary database interrogation system with on-line learning capabilities. the nomad system is a documentary database interrogration system based on a symbolico-connectionist approach.nomad makes use of the synthesis capabilities and flexibility inherent in this type of approach to increase its processing power as compared as compared to existing systems while proposing new operating modes directly accessible to a large number of users:&bull;nomad manages multiple synthetic type views on its documentary contents in the form of neural topographies acting as case-memories as well as elaborated thematic browsing tools.&bull;nomad manages a session memory based on the neural model of the novelty detector with the following three functions: cumulative recording of user need, managing user contradictions and proposing new orientations.&bull;nomad also has extended learning capabilities, enabling it to improve its performance in the long term.in the introduction of this article, we justify the modelling choices made for nomad. the system will then be described in detailed, stressing the new possibilities regarding help in query formulation and improvement in retrieval performance. the evaluation campaign and the numerous evolution perspectives provided by the model are described in the final section.
adaptive information filtering: detecting changes in text streams. the task of information filtering is to classify documents from a stream as either relevant or non-relevant according to a particular user interest with the objective to reduce information load. when using an information filter in an environment that is changing with time, methods for adapting the filter should be considered in order to retain classification accuracy. we favor a methodology that attempts to detect changes and adapts the information filter only if inevitable in order to minimize the amount of user feedback for providing new training data. yet, detecting changes may require costly user feedback as well. this paper describes two methods for detecting changes without user feedback. the first method is based on evaluating an expected error rate, while the second one observes the fraction of classification decisions made with a confidence below a given threshold. further, a heuristics for automatically determining this threshold is suggested and the performance of this approach is experimentally explored as a function of the threshold parameter. some empirical results show that both methods work well in a simulated change scenario with real world data.
on the estimation of frequent itemsets for data streams: theory and experiments. in this paper, we devise a method for the estimation of the true support of itemsets on data streams, with the objective to maximize one chosen criterion among {precision, recall} while ensuring a degradation as reduced as possible for the other criterion. we discuss the strengths, weaknesses and range of applicability of this method that relies on conventional uniform convergence results, yet guarantees statistical optimality from different standpoints.
distributional term representations: an experimental comparison. a number of content management tasks, including term categorization, term clustering, and automated thesaurus generation, view natural language <i>terms</i> (e.g. words, noun phrases) as first-class objects, i.e. as objects endowed with an internal representation which makes them suitable for explicit manipulation by the corresponding algorithms. the information retrieval (ir) literature has traditionally used an extensional (aka <i>distributional</i>) representation for terms according to which a term is represented by the "bag of documents" in which the term occurs. the computational linguistics (cl) literature has independently developed an alternative distributional representation for terms, according to which a term is represented by the "bag of terms" that co-occur with it in some document. this paper aims at discovering which of the two representations is most effective, i.e. brings about higher effectiveness once used in tasks that require terms to be explicitly represented and manipulated. we carry out experiments on (i) a term categorization task, and (ii) a term clustering task; this allows us to compare the two different representations in closely controlled experimental conditions. we report the results of experiments in which we categorize/cluster under 42 different classes the terms extracted from a corpus of more than 65,000 documents. our results show a substantial difference in effectiveness between the two representation styles; we give both an intuitive explanation and an information-theoretic justification for these different behaviours.
indexing and retrieval of scientific literature. the web has greatly improved access to scientific literature. however, scientific articles on the web are largely disorganized, with research articles being spread across archive sites, institution sites, journal sites, and researcher homepages. no index covers all of the available literature, and the major web search engines typically do not index the content of postscript/pdf documents at all. this paper discusses the creation of digital libraries of scientific literature on the web, including the efficient location of articles, full-text indexing of the articles, autonomous citation indexing, information extraction, display of query-sensitive summaries and citation context, hubs and authorities computation, similar document detection, user profiling, distributed error correction, graph analysis, and detection of overlapping documents. the software for the system is available at no cost for non-commercial use.
a self-organized file cabinet. the self-organizing file cabinet is an information retrieval system associated with a user's physical file cabinet. it enhances a physical file cabinet with electronic information about the papers in it. it can remember, organize, update, and help the user find documents contained in the physical file cabinet. the system consists of a module for extracting electronic information about the papers stored in the file cabinet, a module for representing and storing this information in multiple views, and a module that allows a user to interact with this information. the focus of this paper is on the design and evaluation of the self-organized file cabinet.
an automated approach for retrieving hierarchical data from html tables. among the html elements, html tables [rhj98] encapsulate hierarchically structured data (hierarchical data in short) in a tabular structure. html tables do not come with a rigid schema and almost any forms of two-dimensional tables are acceptable according to the html grammar. this relaxation complicates the process of retrieving hierarchical data from html tables. in this paper, we propose an automated approach for retrieving hierarchical data from html tables. the proposed approach constructs the content tree of an html table, which captures the intended hierarchy of the data content of the table, without requiring the internal structure of the table to be known beforehand. also, the user of the content tree does not deal with html tags while retrieving the desired data from the content tree. our approach can be employed by (i) a query language written for retrieving hierarchically structured data, extracted from either the contents of html tables or other sources, (ii) a processor for converting html tables to xml documents, and (iii) a data warehousing repository for collecting hierarchical data from html tables and storing materialized views of the tables. the time complexity of the proposed retrieval approach is proportional to the number of html elements in an html table.
semql: a semantic query language for multidatabase systems. an essential prerequisite to achieving interoperability in multidatabase systems is to be able to identify semantically equivalent or related data items in component databases. another problem in multidatabase systems is allowing users to handle information from different databases that refer to the same realworld entity. in this paper, we provide semantic networks so that multidatabase systems can detect and resolve semantic heterogeneities among component databases. and we provide a semantic query language, semql, to capture the concepts about what users want. it enables users to issue queries to a large number of autonomous databases without prior knowledge of their schemas.
page access scheduling in join processing. the join relational operation is one of the most expensive among database operations. in this study, we consider the problem of scheduling page accesses in join processing. this raises two interesting problems: 1) determining a page access sequence that uses the minimum number of buffer pages without any page reaccesses, and 2) determining a page access sequence that minimizes the number of page reaccesses for a given buffer size. we use a graph model to represent the pages from the relations that contain tuples to be joined, and present new heuristics for the two problems based on the sort-merge join and the simple tid algorithm. our experimental results show that the new heuristics perform well.
semantic caching via query matching for web sources. a semantic caching scheme suitable for wrappers wrapping web sources is presented. since the web sources have typically weaker querying capabilities than conventional databases, existing semantic caching schemes cannot be applied directly. a seamlessly integrated query translation and capability mapping between the wrappers and web sources in semantic caching is described. in addition, an analysis on the match types between the user's input query and cached queries is presented. semantic knowledge acquired from the data can be used to avoid unnecessary access to the web sources by transforming the cache miss to the cache hit. a polynomial time algorithm based on the proposed query matching technique is presented to find the best matched query in the cache. experimental results reveal the effectiveness of the proposed semantic caching scheme.
efficient evaluation of multiple queries on streaming xml data. traditionally, xml documents are processed at where they are stored. this allows the query processor to exploit pre-computed data structures (e.g., index) to retrieve the desired data efficiently. however, this mode of processing is not suitable for many applications where the documents are frequently updated. in such situations, efficient evaluation of multiple queries over streaming xml documents becomes important. this paper introduces a new operator, mqx-scan, which efficiently evaluates multiple queries with a single pass on streaming xml data. to facilitate matching, mqx-scan utilizes templates containing paths that have been traversed to match regular path expression patterns in a pool of queries. results of the experiments demonstrate the efficiency and scalability of the mqx-scan operator.
training a selection function for extraction. in this paper we compare performance of several heuristics in generating informative generic/query-oriented extracts for newspaper articles in order to learn how topic prominence affects the performance of each heuristic. we study how different query types can affect the performance of each heuristic and discuss the possibility of using machine learning algorithms to automatically learn good combination functions to combine several heuristics. we also briefly describe the design, implementation, and performance of a multilingual text summarization system summarist.
information integration with attributio support for corporate profiles. the proliferation of electronically available data within large organizations as well as publicly available data (e.g. over the world wide web) poses challenges for users who wish to efficiently interact with and integrate multiple heterogeneous sources. this paper presents ci3, a corporate information integrator, which applies xml as a tool to facilitate data mediation and integration amongst heterogeneous sources in the context of financial analysts creating corporate profiles. sources include lotus notes, relational databases, and the world wide web. ci3 applies a unified xml data model to automate integration. by preserving metadata about the source of each datum in the integrated result set, ci3 supports source attribution. users may trace the attribution metadata from the result back to the underlying sources and leverage their expertise in interpreting the data and, if necessary, use their judgment in assessing the authenticity and veracity of results. we present a functional overview of ci3, its system architecture including the xml data model, and the integration procedures. we conclude by reflecting on lessons learned.