waashk/dblp huggingface dataset:
sql extension for exploring multiple tables. the standard sql assumes that the users are aware of all tables and their schemas to write queries. this assumption may be valid when the users deal with a relatively small number of tables, but writing a sql query on a large number of tables is often challenging; (1) the users do not know what tables are relevant to their query, (2) it is too cumbersome to explicitly list tens of (or even hundreds of) relevant tables in the from clause and (3) the schemas of those tables are not identical. we now propose an intuitive yet powerful extension to sql that helps users explore and aggregate information spread over a large number of tables.
snif tool: sniffing for patterns in continuous streams. continuous time-series sequence matching, specifically, matching a numeric live stream against a set of redefined pattern sequences, is critical for domains ranging from fire spread tracking to network traffic monitoring. while several algorithms exist for similarity matching of static time-series data, matching continuous data poses new, largely unsolved challenges including online real-time processing requirements and system resource limitations for handling infinite streams. in this work, we propose a novel live stream matching framework, called n-snippet indices framework (in short, snif), to tackle these challenges. snif employs snippets as the basic unit for matching streaming time-series. the insight is to perform the matching at two levels of granularity: bag matching of subsets of snippets of the live stream against prefixes of the patterns, and order checking for maintaining successive candidate snippet bag matches. we design a two-level index structure, called snif index, which supports these two modes of matching. we propose a family of online two-level prefix matching algorithms that trade off between result accuracy and response time. the effectiveness of snif to detect patterns has been thoroughly tested through experiments using real datasets from the domains of fire monitoring and sensor motes. in this paper, we also present a study of snif's performance, accuracy and tolerance to noise compared against those of the state-of-the-art continuous query with prediction (cqp) approach.
using structured text for large-scale attribute extraction. we propose a weakly-supervised approach for extracting class attributes from structured text available within web documents. the overall precision of the extracted attributes is around 30% higher than with previous methods operating on web documents. in addition to attribute extraction, this approach also automatically identifies values for a subset of the extracted class attributes.
integrating web query results: holistic schema matching. the emergence of numerous data sources online has presented a pressing need for more automatic yet accurate data integration techniques. for the data returned from querying such sources, most works focus on how to extract the embedded structured data more accurately. however, to eventually provide an integrated access to these query results, a last but not least step is to combine the extracted data coming from different sources. a critical task is finding the correspondence of the data fields between the sources - a problem well known as schema matching. query results are a small and biased sample set of instances obtained from sources; the obtained schema information is thus very implicit and incomplete, which often prevents existing schema matching approaches from performing effectively. in this paper, we develop a novel framework for understanding and effectively supporting schema matching on such instance-based data, especially for integrating multiple sources. we view discovering matching as constructing a more complete domain schema that best describes the input data. with this conceptual view, we can leverage various data instances and observed regularities seamlessly with holistic, multiple-source schema matching to achieve more accurate matching results. our experiments show that our framework consistently outperforms baseline pairwise and clustering-based approaches (raising f-measure from 50-89% to 89-94%) and works uniformly well for the surveyed domains.
fast spatial co-location mining without cliqueness checking. in this paper, we propose a novel, spatial co-location mining algorithm which automatically generates co-located spatial features without generating any non-clique candidates at each level. subsequently our algorithm is more efficient than other existing level-wise co-location algorithms because no cliqueness checking is performed in our algorithm. in addition, our algorithm produces a smaller number of co-location candidates than the other existing algorithms.
handling implicit geographic evidence for geographic ir. most geographic information retrieval systems depend on the detection and disambiguation of place names in documents, assuming that the documents with a specific geographic scope contain explicit place names in the text that are strongly related to the document scopes. however, some non-geographic names such as companies, monuments or sport events, may also provide indirect relevant evidence that can significantly contribute to the assignment of geographic scopes to documents. in this paper, we analyze the amount of implicit and explicit geographic evidence in newspaper documents, and measure its impact on geographic information retrieval by evaluating the performance of a retrieval system using the geoclef evaluation data.
data weaving: scaling up the state-of-the-art in data clustering. the enormous amount and dimensionality of data processed by modern data mining tools require effective, scalable unsupervised learning techniques. unfortunately, the majority of previously proposed clustering algorithms are either effective or scalable. this paper is concerned with information-theoretic clustering (itc) that has historically been considered the state-of-the-art in clustering multi-dimensional data. most existing itc methods are computationally expensive and not easily scalable. those few itc methods that scale well (using, e.g., parallelization) are often outperformed by the others, of an inherently sequential nature. first, we justify this observation theoretically. we then propose data weaving - a novel method for parallelizing sequential clustering algorithms. data weaving is intrinsically multi-modal - it allows simultaneous clustering of a few types of data (modalities). finally, we use data weaving to parallelize multi-modal itc, which results in proposing a powerful dataloom algorithm. in our experimentation with small datasets, dataloom shows practically identical performance compared to expensive sequential alternatives. on large datasets, however, dataloom demonstrates significant gains over other parallel clustering methods. to illustrate the scalability, we simultaneously clustered rows and columns of a contingency table with over 120 billion entries.
link privacy in social networks. we consider a privacy threat to a social network in which the goal of an attacker is to obtain knowledge of a significant fraction of the links in the network. we formalize the typical social network interface and the information about links that it provides to its users in terms of lookahead. we consider a particular threat in which an attacker subverts user accounts to gain information about local neighborhoods in the network and pieces them together in order to build a global picture. we analyze, both experimentally and theoretically, the number of user accounts an attacker would need to subvert for a successful attack, as a function of his strategy for choosing users whose accounts to subvert and a function of the lookahead provided by the network. we conclude that such an attack is feasible in practice, and thus any social network that wishes to protect the link privacy of its users should take great care in choosing the lookahead of its interface, limiting it to 1 or 2, whenever possible.
mining term association patterns from search logs for effective query reformulation. search engine logs are an emerging new type of data that offers interesting opportunities for data mining. existing work on mining such data has mostly attempted to discover knowledge at the level of queries (e.g., query clusters). in this paper, we propose to mine search engine logs for patterns at the level of terms through analyzing the relations of terms inside a query. we define two novel term association patterns (i.e., context-sensitive term substitutions and term additions) and propose new methods for mining such patterns from search engine logs. these two patterns can be used to address the mis-specification and under-specification problems of ineffective queries. experiment results on real search engine logs show that the mined context-sensitive term substitutions can be used to effectively reword queries and improve their accuracy, while the mined context-sensitive term addition patterns can be used to support query refinement in a more effective way.
beyond the session timeout: automatic hierarchical segmentation of search topics in query logs. most analysis of web search relevance and performance takes a single query as the unit of search engine interaction. when studies attempt to group queries together by task or session, a timeout is typically used to identify the boundary. however, users query search engines in order to accomplish tasks at a variety of granularities, issuing multiple queries as they attempt to accomplish tasks. in this work we study real sessions manually labeled into hierarchical tasks, and show that timeouts, whatever their length, are of limited utility in identifying task boundaries, achieving a maximum precision of only 70%. we report on properties of this search task hierarchy, as seen in a random sample of user interactions from a major web search engine's log, annotated by human editors, learning that 17% of tasks are interleaved, and 20% are hierarchically organized. no previous work has analyzed or addressed automatic identification of interleaved and hierarchically organized search tasks. we propose and evaluate a method for the automated segmentation of users' query streams into hierarchical units. our classifiers can improve on timeout segmentation, as well as other previously published approaches, bringing the accuracy up to 92% for identifying fine-grained task boundaries, and 89-97% for identifying pairs of queries from the same task when tasks are interleaved hierarchically. this is the first work to identify, measure and automatically segment sequences of user queries into their hierarchical structure. the ability to perform this kind of segmentation paves the way for evaluating search engines in terms of user task completion.
in the development of a spanish metamap. metamap is an online application that allows mapping text to umls metathesaurus concepts, which is very useful interoperability among different languages and systems within the biomedical domain. metamap transfer (mmtx) is a java program that makes metamap available to biomedical researchers. currently there is no spanish version of metamap, which difficults the use of umls metathesaurus to extract concepts from spanish biomedical texts. our ongoing research is mainly focused on using biomedical concepts for cross-lingual text classification and retrieval [3]. in this context the use of concepts instead of bag of words representation allows us to face text classification tasks abstracting from the language [4]. in this paper we evaluate the possibility of combining automatic translation techniques with the use of biomedical ontologies to produce an english text that can be processed by mmtx.
semi-supervised metric learning by maximizing constraint margin. distance metric learning is an old problem that has been researched in the supervised learning field for a very long time. in this paper, we consider the problem of learning a proper distance metric under the guidance of some weak supervisory information. specifically, those information are in the form of pairwise constraints which specify whether a pair of data points are in the same class (must link constraints) or in the different classes (cannot link constraints). given those constraints, our algorithm aims to learn a distance metric under which the points with must link constraints are pushed as close as possible, while simultaneously the points with cannot link constraints are pulled away as far as possible. finally the experimental results are presented to show the effectiveness of our method.
coreex: content extraction from online news articles. we developed and tested a heuristic technique for extracting the main article from news site web pages. we construct the dom tree of the page and score every node based on the amount of text, the number of links it contains and additional heuristics. the method is site-independent and does not use any language-based features. we tested our algorithm on a set of 1120 news article pages from 27 domains. our algorithm achieved over 97% precision and 98% recall, and an average processing speed of under 15ms per page.
a system for finding biological entities that satisfy certain conditions from texts. finding biological entities (such as genes or proteins) that satisfy certain conditions from texts is an important and challenging task in biomedical information retrieval and text mining. it is essential for many biomedical applications, such as drug discovery which normally requires collecting existing scientific facts from documents. this paper presents an effective ir system for this task, in which 1) domain knowledge is incorporated to improve retrieval effectiveness; 2) query expansion with related concepts on multiple semantic levels is employed; 3) a gene symbol disambiguation technique is implemented. we evaluated these techniques and examined two different concept-based ir models. experiments based upon the proposed framework yield significant improvement (22% for automatic and 16.7% for non-automatic) over the best reported results of passage retrieval in the genomics track of trec 2007.
closing the loop in webpage understanding. the two most important tasks in information extraction from the web are webpage structure understanding and natural language sentences processing. however, little work has been done toward an integrated statistical model for understanding webpage structures and processing natural language sentences within the html elements. our recent work on webpage understanding introduces a joint model of hierarchical conditional random fields (hcrfs) and extended semi-markov conditional random fields (semi-crfs) to leverage the page structure understanding results in free text segmentation and labeling. in this top-down integration model, the decision of the hcrf model could guide the decision making of the semi-crf model. however, the drawback of the top-down integration strategy is also apparent, i.e., the decision of the semi-crf model could not be used by the hcrf model to guide its decision making. this paper proposed a novel framework called webnlp, which enables bidirectional integration of page structure understanding and text understanding in an iterative manner. we have applied the proposed framework to local business entity extraction and chinese person and organization name extraction. experiments show that the webnlp framework achieved significantly better performance than existing methods.
comparing metrics across trec and ntcir: the robustness to system bias. test collections are growing larger, and relevance data constructed through pooling are suspected of becoming more and more incomplete and biased. several studies have used evaluation metrics specifically designed to handle this problem, but most of them have only examined the metrics under incomplete but unbiased conditions, using random samples of the original relevance data. this paper examines nine metrics in a more realistic setting, by reducing the number of pooled systems. even though previous work has shown that metrics based on a condensed list, obtained by removing all unjudged documents from the original ranked list, are effective for handling very incomplete but unbiased relevance data, we show that these results do not hold in the presence of system bias. in our experiments using trec and ntcir data, we first show that condensed-list metrics overestimate new systems while traditional metrics underestimate them, and that the overestimation tends to be larger than the underestimation. we then show that, when relevance data is heavily biased towards a single team or a few teams, the condensed-list versions of average precision (ap), q-measure (q) and normalised discounted cumulative gain (ndcg), which we call ap', q' and ndcg', are not necessarily superior to the original metrics in terms of discriminative power, i.e., the overall ability to detect pairwise statistical significance. nevertheless, even under system bias, ap' and q' are generally more discriminative than bpref and the condensed-list version of rank-biased precision (rbp), which we call rbp'.
a densitometric approach to web page segmentation. web page segmentation is a crucial step for many applications in information retrieval, such as text classification, de-duplication and full-text search. in this paper we describe a new approach to segment html pages, building on methods from quantitative linguistics and strategies borrowed from the area of computer vision. we utilize the notion of text-density as a measure to identify the individual text segments of a web page, reducing the problem to solving a 1d-partitioning task. the distribution of segment-level text density seems to follow a negative hypergeometric distribution, described by frumkina's law. our extensive evaluation confirms the validity and quality of our approach and its applicability to the web.
a georeferencing multistage method for locating geographic context in web search. the geographic scope of web pages is becoming an essential dimension of web search, especially for mobile users. this paper shows a multistage method for assigning a geographic focus to web pages (georeferencing) according to their text contents. we suggest several heuristics for the disambiguation toponyms and a scoring procedure for focus determination. furthermore, we provide an experimental methodology for evaluating the accuracy. finally, we obtained promising results of over 70% accuracy with a city-level resolution.
transfer learning from multiple source domains via consensus regularization. recent years have witnessed an increased interest in transfer learning. despite the vast amount of research performed in this field, there are remaining challenges in applying the knowledge learnt from multiple source domains to a target domain. first, data from multiple source domains can be semantically related, but have different distributions. it is not clear how to exploit the distribution differences among multiple source domains to boost the learning performance in a target domain. second, many real-world applications demand this transfer learning to be performed in a distributed manner. to meet these challenges, we propose a consensus regularization framework for transfer learning from multiple source domains to a target domain. in this framework, a local classifier is trained by considering both local data available in a source domain and the prediction consensus with the classifiers from other source domains. in addition, the training algorithm can be implemented in a distributed manner, in which all the source-domains are treated as slave nodes and the target domain is used as the master node. to combine the training results from multiple source domains, it only needs share some statistical data rather than the full contents of their labeled data. this can modestly relieve the privacy concerns and avoid the need to upload all data to a central location. finally, our experimental results show the effectiveness of our consensus regularization learning.
speed up semantic search in p2p networks. peer-to-peer architectures become popular in modern massively distributed systems, which are often in very large scale and contain a huge volume of heterogeneous data. to facilitate the information retrieval process in p2p networks, we consider semantic search approach, where syntax-based queries are shipped to peers based on semantic correlations. motivated by an interesting experience in web information retrieval, we propose a novel ontology-based scheme to measure similarity of peer interests accurately and consistently in a decentralized way, and group peers under a scalable hierarchical overlay network. given queries, our approach either floods them within local peer groups or guides them towards remote groups based on the similarity of interests. our work overcomes the limitations of the existing p2p hybrid-search approaches by avoiding costly data popularity measurement. performance evaluation and comparison against baseline algorithms show that our approach provides a better solution for information retrieval in large-scale p2p networks.
pattern-based semantic class discovery with multi-membership support. a semantic class is a collection of items (words or phrases) sharing common semantic properties. this paper proposes an approach to constructing one or multiple semantic classes for an input item. two challenges are addressed: multi-membership, and noise-tolerance.
redus: finding reducible subspaces in high dimensional data. finding latent patterns in high dimensional data is an important research problem with numerous applications. the most well known approaches for high dimensional data analysis are feature selection and dimensionality reduction. being widely used in many applications, these methods aim to capture global patterns and are typically performed in the full feature space. in many emerging applications, however, scientists are interested in the local latent patterns held by feature subspaces, which may be invisible via any global transformation. in this paper, we investigate the problem of finding strong linear and nonlinear correlations hidden in feature subspaces of high dimensional data. we formalize this problem as identifying reducible subspaces in the full dimensional space. intuitively, a reducible subspace is a feature subspace whose intrinsic dimensionality is smaller than the number of features. we present an effective algorithm, redus, for finding the reducible subspaces. two key components of our algorithm are finding the overall reducible subspace, and uncovering the individual reducible subspaces from the overall reducible subspace. a broad experimental evaluation demonstrates the effectiveness of our algorithm.
ghost: an effective graph-based framework for name distinction. name ambiguity stems from the fact that many people or objects share identical names. in this paper, we focus on investigating the problem in digital libraries to distinguish publications written by authors with identical names. we present an effective graph-based framework, ghost (abbr. graph-based framework for name distinction), to solve the problem systematically. we evaluated the framework on the real dblp dataset, and the experimental results show that ghost outperforms the state-of-the-art method.