waashk/dblp huggingface dataset:
kernel latent semantic analysis using an information retrieval based kernel. hidden term relationships can be found within a document collection using latent semantic analysis (lsa) and can be used to assist in information retrieval. lsa uses the inner product as its similarity function, which unfortunately introduces bias due to document length and term rarity into the term relationships. in this article, we present the novel kernel based lsa method, which uses separate document and query kernel functions to compute document and query similarities, rather than the inner product. we show that by providing an appropriate kernel function, we are able to provide a better fit of our data and hence produce more effective term relationships.
an integrated discriminative probabilistic approach to information extraction. probabilistic graphical models for sequence data enable us to effectively deal with inherent uncertainty in many real-world domains. however, they operate on a mostly propositional level. logic approaches, on the other hand, can compactly represent a wide variety of knowledge, especially first-order ones, but treat uncertainty only in limited ways. therefore, combining probability and first-order logic is highly desirable for information extraction which requires uncertainty modeling as well as dependency and deeper knowledge representation. in this paper, we model both segmentations in observation sequence and relations of segments simultaneously in our proposed integrated discriminative probabilistic framework. we propose the metropolis-hastings, a markov chain monte carlo (mcmc) algorithm for approximate bayesian inference to find the maximum a posteriori assignment of all the variables of this model. this integrated model has several advantages over previous probabilistic graphical models, and it offers a great capability of extracting implicit relations and new relation discovery for relation extraction from encyclopedic documents, and capturing sub-structures in named entities for named entity recognition. we performed extensive experiments on the above two well-established information extraction tasks, illustrating the feasibility and promise of our approach.
voting in social networks. a voting system is a set of rules that a community adopts to take collective decisions. in this paper we study voting systems for a particular kind of community: electronically mediated social networks. in particular, we focus on delegative democracy (a.k.a. proxy voting) that has recently received increased interest for its ability to combine the benefits of direct and representative systems, and that seems also perfectly suited for electronically mediated social networks. in such a context, we consider a voting system in which users can only express their preference for one among the people they are explicitly connected with, and this preference can be propagated transitively, using an attenuation factor. we present this system and we study its properties. we also take into consideration the problem of missing votes, which is particularly relevant in online networks, as some recent case shows. our experiments on real-world networks provide interesting insight into the significance and stability of the results obtained with the suggested voting system.
a signal-to-noise approach to score normalization. score normalization is indispensable in distributed retrieval and fusion or meta-search where merging of result-lists is required. distributional approaches to score normalization with reference to relevance, such as binary mixture models like the normal-exponential, suffer from lack of universality and troublesome parameter estimation especially under sparse relevance. we develop a new approach which tackles both problems by using aggregate score distributions without reference to relevance, and is suitable for uncooperative engines. the method is based on the assumption that scores produced by engines consist of a signal and a noise component which can both be approximated by submitting well-defined sets of artificial queries to each engine. we evaluate in a standard distributed retrieval testbed and show that the signal-to-noise approach yields better results than other distributional methods. as a significant by-product, we investigate query-length distributions.
an analysis framework for search sequences. in this paper we present a general framework to study sequences of search activities performed by a user. our framework provides (i) a vocabulary to discuss types of features, models, and tasks, (ii) straightforward feature re-use across problems, (iii) realistic baselines for many sequence analysis tasks we study, and (iv) a simple mechanism to develop baselines for sequence analysis tasks beyond those studied in this paper. using this framework we study a set of fourteen sequence analysis tasks with a range of features and models. while we show that most tasks benefit from features based on recent history, we also identify two categories of "sequence-resistant" tasks for which simple classes of local features perform as well as richer features and models.
spatio-temporal association rule mining framework for real-time sensor network applications. in this paper, we present a data mining framework to estimate missing or corrupted data in sensor network applications - a frequently occurring phenomenon in this domain. the framework is naturally germane to the spatio-temporal analysis of relational data stream evolution. our method utilizes association rules to capture spatio-temporal correlations in multivariate, dynamically evolving, and unbounded sensor data streams. existing approaches that tackled this problem do not account for the multi-dimensionality of the node data and their relationship; furthermore they entail simplistic and/or premature assumptions on the temporal and spatial factors to overcome the complexity of the streaming environment. our technique, called mining autonomously spatio-temporal environmental rules (master), comprehensively formulates the problem of mining patterns in sensor data streams, and yet remains provably adaptive to bounded time and space costs while probabilistically assuring a bounded estimation error. simulation experiments show master's efficiency in terms of overhead as well as the quality of estimation.
structure-aware indexing for keyword search in databases. most of existing methods of keyword search over relational databases find the steiner trees composed of relevant tuples as the answers. they identify the steiner trees by discovering the rich structural relationships between tuples, and neglect the fact that such structural relationships can be pre-computed and indexed. tuple units that are composed of most relevant tuples are proposed to address this problem. tuple units can be precomputed and indexed. existing methods identify a single tuple unit to answer keyword queries. they, however, may involve false negatives as in many cases a single tuple unit cannot answer a keyword query. instead, multiple tuple units should be integrated to answer keyword queries. to address this problem, in this paper, we study how to integrate multiple related tuple units to effectively answer keyword queries. we devise novel indices and incorporate the structural relationships between different tuple units into the indices. we use the indices to efficiently and progressively identify the top-k relevant answers. we have implemented our method in real database systems, and the experimental results show that our approach achieves high search efficiency and accuracy, and outperforms state-of-the-art methods significantly.
experiments on pattern-based relation learning. relation extraction is the task of extracting semantic relations - such as synonymy or hypernymy - between word pairs from corpus data. past work in relation extraction has concentrated on manually creating templates to use in directly extracting word pairs for a given semantic relation from corpus text. recently, there has been a move towards using machine learning to automatically learn these patterns. we build on this research by running experiments investigating the impact of corpus type, corpus size and different parameter settings on learning a range of lexical relations.
xcfs: an xml documents clustering approach using both the structure and the content. this paper introduces a clustering approach, xml clustering using frequent substructures (xcfs) that considers both the structural and the content information of xml documents in clustering. xcfs uses frequent substructures in the form of a novel representation, closed frequent embedded (cfe) subtrees to constrain the content in the clustering process. the empirical analysis ascertains that xcfs can effectively cluster even very large xml datasets and outperforms other existing methods.
characterizing, constructing and managing resource usage profiles of system s applications: challenges and experience. we describe the challenges of characterizing, constructing and managing the usage profiles of system s applications. a running system s application is a directed graph with software processing elements(pes) as vertices and data streams as edges connecting the pes. the resource usage of each pe is a critical input to the runtime scheduler for proper resource allocation. we represent the resource usage of pes in terms of resource functions (rfs) that are used by the system s scheduler, with one rf per resource per pe. the first challenge is that it is difficult to build good rfs that can accurately predict the resource usage of a pe because the pes perform arbitrary computations. a second set of challenges arises in managing the rfs and performance data so that we can apply them for pes that are re-run or reused by the same or different applications or users. we report our experience in overcoming these challenges. specifically, we present an empirical characterization of pe rfs from several real streaming applications running in a system s testbed. this indicates that our simple models of resource usage that build on the data-flow nature of the underlying application can be effective, even for complex pes. to illustrate our methodology, we evaluate and analyze the performance of these applications as a function of the quality of our resource profile models. the system automatically learns the models from the raw metrics data collected from running pes. we describe our approach to managing the metrics and rf models, which allows us to construct generalizable rfs and eliminates the learning time for new pes by intelligently storing and reusing the metrics data.
dynamic hyperparameter optimization for bayesian topical trend analysis. this paper presents a new bayesian topical trend analysis. we regard the parameters of topic dirichlet priors in latent dirichlet allocation as a function of document timestamps and optimize the parameters by a gradient-based algorithm. since our method gives similar hyperparameters to the documents having similar timestamps, topic assignment in collapsed gibbs sampling is affected by timestamp similarities. we compute tfidf-based document similarities by using a result of collapsed gibbs sampling and evaluate our proposal by link detection task of topic detection and tracking.
to obtain orthogonal feature extraction using training data selection. feature extraction is an effective tool in data mining and machine learning. many feature extraction methods have been investigated recently. however, few methods can achieve orthogonal components. non-orthogonal components distort the metric structure of original data space and contain reductant information. in this paper, we propose a feature extraction method, named as incremental orthogonal basis analysis (ioba), to cope with the challenging endeavors. first, ioba learns orthogonal components for original data, not only theoretically but also numerically. second, an innovative way of training data selection is proposed. this selection scheme helps ioba pick up numerically orthogonal components from training patterns. third, by designing a self-adaptive threshold technique, no prior knowledge about the number of components is necessary to use ioba. moreover, without solving eigenvalue and eigenvector problems, ioba not only saves large computing loads, but also avoids ill-conditioned problems. results of experiments show the efficiency of the proposed ioba.
comprehenrank: estimating comprehension in classroom by absorbing random walks on a cognitive graph. this paper develops a graph-theoretic framework for estimating comprehension in classroom. to deal with imprecise data gathered in classroom, we propose multi-step comprehension propagation over a semantic graph. random walks on the graph measure students' comprehension with probabilities absorbed at student nodes.
a collaborative filtering approach to ad recommendation using the query-ad click graph. search engine logs contain a large amount of click-through data that can be leveraged as soft indicators of relevance. in this paper we address the sponsored search retrieval problem which is to find and rank relevant ads to a search query. we propose a new technique to determine the relevance of an ad document for a search query using click-through data. the method builds on a collaborative filtering approach to discover new ads related to a query using a click graph. it is implemented on a graph with several million edges and scales to larger sizes easily. the proposed method is compared to three different baselines that are state-of-the-art for a commercial search engine. evaluations on editorial data indicate that the model discovers many new ads not retrieved by the baseline methods. the ads from the new approach are on average of better quality than the baselines.
stochastic gradient boosted distributed decision trees. stochastic gradient boosted decision trees (gbdt) is one of the most widely used learning algorithms in machine learning today. it is adaptable, easy to interpret, and produces highly accurate models. however, most implementations today are computationally expensive and require all training data to be in main memory. as training data becomes ever larger, there is motivation for us to parallelize the gbdt algorithm. parallelizing decision tree training is intuitive and various approaches have been explored in existing literature. stochastic boosting on the other hand is inherently a sequential process and have not been applied to distributed decision trees. in this work, we present two different distributed methods that generates exact stochastic gbdt models, the first is a mapreduce implementation and the second utilizes mpi on the hadoop grid environment.
exploiting internal and external semantics for the clustering of short texts using world knowledge. clustering of short texts, such as snippets, presents great challenges in existing aggregated search techniques due to the problem of data sparseness and the complex semantics of natural language. as short texts do not provide sufficient term occurring information, traditional text representation methods, such as ``bag of words" model, have several limitations when directly applied to short texts tasks. in this paper, we propose a novel framework to improve the performance of short texts clustering by exploiting the internal semantics from original text and external concepts from world knowledge. the proposed method employs a hierarchical three-level structure to tackle the data sparsity problem of original short texts and reconstruct the corresponding feature space with the integration of multiple semantic knowledge bases -- wikipedia and wordnet. empirical evaluation with reuters and real web dataset demonstrates that our approach is able to achieve significant improvement as compared to the state-of-the-art methods.
efficient and reliable merging of xml documents. many knowledge-based processes rely on xml-based office documents. up to now, versioning and merging xml documents was a difficult and error-prone task, mostly done manually. the support by tools is still in its infancy. we have presented a novel approach to compare and merge xml documents in a reliable way using context fingerprints. in this demonstration, we show the application of our toolset to odf documents using a new, simple, and user-friendly graphical user-interface.
semantic queries in databases: problems and challenges. supporting semantic queries in relational databases is essential to many advanced applications. recently, with the increasing use of ontology in various applications, the need for querying relational data together with its related ontology has become more urgent. in this paper, we identify and discuss the problem of querying relational data with its ontologies. two fundamental challenges make the problem interesting. first, it is extremely difficult to express queries against graph structured ontology in the relational query language sql, and second, in many cases where data and its related ontology are complicated, queries are usually not precise, that is, users often have only a vague notion, rather than a clear understanding and definition, of what they query for. we outline a query-by-example approach that enables us to support semantic queries in relational databases with ease. instead of endeavoring to incorporate ontology into relational form and create new language constructs to express such queries, we ask the user to provide a small number of examples that satisfy the query she has in mind. using these examples as seeds, the system infers the exact query automatically, and the user is therefore shielded from the complexity of interfacing with the ontology.
hypersum: hypergraph based semi-supervised sentence ranking for query-oriented summarization. graph based sentence ranking algorithms such as pagerank and hits have been successfully used in query-oriented summarization. with these algorithms, the documents to be summarized are often modeled as a text graph where nodes represent sentences and edges represent pairwise similarity relationships between two sentences. a deficiency of conventional graph modeling is its incapability of naturally and effectively representing complex group relationships shared among multiple objects. simply squeezing complex relationships into pairwise ones will inevitably lead to loss of information which can be useful for ranking and learning. in this paper, we propose to take advantage of hypergraph, i.e. a generalization of graph, to remedy this defect. in a text hypergraph, nodes still represent sentences, yet hyperedges are allowed to connect more than two sentences. with a text hypergraph, we are thus able to integrate both group relationships formulated among multiple sentences and pairwise relationships formulated between two sentences in a unified framework. as essential work, it is first addressed in the paper that how a text hypergraph can be built for summarization by applying clustering techniques. then, a hypergraph based semi-supervised sentence ranking algorithm is developed for query-oriented extractive summarization, where the influence of query is propagated to sentences through the structure of the constructed text hypergraph. when evaluated on duc data sets, performance of the proposed approach is remarkable.
real-word spelling correction using google web 1tn-gram data set. we present a method for correcting real-word spelling errors using the google web 1t n-gram data set and a normalized and modified version of the longest common subsequence (lcs) string matching algorithm. our method is focused mainly on how to improve the correction recall (the fraction of errors corrected) while keeping the correction precision (the fraction of suggestions that are correct) as high as possible. evaluation results on a standard data set show that our method performs very well.
domain driven data mining to improve promotional campaign roi and select marketing channels. the trading activities of materials retail is concerned with an extremely competitive market. however, business people are not well informed about how to proceed and what to do during marketing activities. data mining methods could be interesting to generate substantial profits for decision makers and to optimize the choice of different marketing activities. in this paper, we propose an actionable knowledge discovery methodology, for one-to-one marketing, which allows to contact the right customer through the right communication channel. this methodology first requires a measurement of the tendency for the customers to purchase a given item, and second requires an optimization of the return on investment by selecting the most effective communication channels for attracting these customers. our methodology has been applied to the vm matériaux company. thanks to the collaboration between data miners and decision makers, we present a domain-driven view of knowledge discovery satisfying real business needs to improve the efficiency and outcome of several promotional marketing campaigns.
a fast and simple method for extracting relevant content from news webpages. we propose nce, an efficient algorithm to identify and extract relevant content from news webpages. we define relevant as the textual sections that more objectively describe the main event in the article. this includes the title and the main body section, and excludes comments about the story and presentation elements. our experiments suggest that nce is competitive, in terms of extraction quality, with the best methods available in the literature. it achieves f1 = 90.7% in our test corpus containing 324 news webpages from 22 sites. the main advantages of our method are its simplicity and its computational performance. it is at least an order of magnitude faster than methods that use visual features. this characteristic is very suitable for applications that process a large number of pages.
3se: a semi-structured search engine for heterogeneous data in graph model. as the ubiquitous interplay of structured, semi-structured and unstructured data from different sources, neither db-style structured query requiring knowledge of full schema and complex language, nor ir-style keyword search ignoring latent structures, can satisfy users. in this paper, we present a novel semi-structured search engine (3se) that provides easy, flexible, precise and rapid access to heterogeneous data represented by a semi-structured graph model. by using an intuitive 3se query language (3sql), users are able to pose queries on heterogeneous data in a varying degree of structural constraint according to their knowledge of schema. 3se evaluates 3sql queries as the top-k answers composed of "logical units" and relationship paths between them, and thus can extract meaningful information even if the query conditions are vague, ambiguous, and inaccurate.
maximal metric margin partitioning for similarity search indexes. we propose a partitioning scheme for similarity search indexes that is called maximal metric margin partitioning (mmmp). mmmp divides the data on the basis of its distribution pattern, especially for the boundaries of clusters. a partitioning surface created by mmmp is likely to be at maximum distances from the two cluster boundaries. mmmp is the first similarity search index approach to focus on partitioning surfaces and data distribution patterns. we also present an indexing scheme, named the mmmp-index, which uses mmmp and small ball partitioning. the mmmp-index prunes many objects that are not relevant to a query, and it reduces the query execution cost. our experimental results show that mmmp effectively indexes clustered data and reduces the search cost. for clustered vector data, the mmmp-index reduces the computational cost to less than two thirds that of comparable schemes.