waashk/dblp huggingface dataset:
topic and keyword re-ranking for lda-based topic modeling. topic-based text summaries promise to help average users quickly understand a text collection and derive insights. recent research has shown that the latent dirichlet allocation (lda) model is one of the most effective approaches to topic analysis. however, the lda-based results may not be ideal for human understanding and consumption. in this paper, we present several topic and keyword re-ranking approaches that can help users better understand and consume the lda-derived topics in their text analysis. our methods process the lda output based on a set of criteria that model a user's information needs. our evaluation demonstrates the usefulness of the methods in summarizing several large-scale, real world data sets.
a code generation approach to optimizing high-performance distributed data stream processing. we present a code-generation-based optimization approach to bringing performance and scalability to distributed stream processing applications. we express stream processing applications using an operator-based, stream-centric language called spade, which supports composing distributed data flow graphs out of toolkits of type-generic operators. a major challenge in building such applications is to find an effective and flexible way of mapping the logical graph of operators into a physical one that can be deployed on a set of distributed nodes. this involves finding how best operators map to processes and how best processes map to computing nodes. in this paper, we take a two-stage optimization approach, where an instrumented version of the application is first generated by the spade compiler to profile and collect statistics about the processing and communication characteristics of the operators within the application. in the second stage, the profiling information is fed to an optimizer to come up with a physical data flow graph that is deployable across nodes in a computing cluster. this approach not only creates highly optimized applications that are tailored to the underlying computing and networking infrastructure, but also makes it possible to re-target the application to a different hardware setup by simply repeating the optimization step and re-compiling the application to match the physical flow graph produced by the optimizer. using real-world applications, from diverse domains such as finance and radio-astronomy, we demonstrate the effectiveness of our approach on system s -- a large-scale, distributed stream processing platform.
consistent on-line classification of dbs workload events. an important goal of self-managing databases is the autonomic adaptation of the database configuration to evolving workloads. however, the diversity of sql statements in real-world workloads typically causes the required analysis overhead to be prohibitive for a continuous workload analysis. the workload classification presented in this paper reduces the workload analysis overhead by grouping similar workload events into classes. our approach employs clustering techniques based upon a general distance function for dbs workload events. to be applicable for a continuous workload analysis, our workload classification specifically addresses a stream-based, lightweight operation, a controllable loss of quality, and self-management.
efficient joins with compressed bitmap indexes. we present a new class of adaptive algorithms that use compressed bitmap indexes to speed up evaluation of the range join query in relational databases. we determine the best strategy to process a join query based on a fast sub-linear time computation of the join selectivity (the ratio of the number of tuples in the result to the total number of possible tuples). in addition, we use compressed bitmaps to represent the join output compactly: the space requirement for storing the tuples representing the join of two relations is asymptotically bounded by min(h; n.cb), where h is the number of tuple pairs in the result relation, n is the number of tuples in the smaller of the two relations, and cb is the cardinality of the larger column being joined. we present a theoretical analysis of our algorithms, as well as experimental results on large-scale synthetic and real data sets. our implementations are efficient, and consistently outperform well-known approaches for a range of join selectivity factors. for instance, our count-only algorithm is up to three orders of magnitude faster than the sort-merge approach, and our best bitmap index-based algorithm is 1.2x-80x faster than the sort-merge algorithm, for various query instances. we achieve these speedups by exploiting several inherent performance advantages of compressed bitmap indexes for join processing: an implicit partitioning of the attributes, space-efficiency, and tolerance of high-cardinality relations.
clustering object moving patterns for prediction-based object tracking sensor networks. prior works have shown that probabilistic suffix trees (pst) could predict accurately the moving behaviors of objects for prediction-based object tracking sensor networks. however, maintaining psts for objects incurs a considerable amount of storage spaces for resource-constrained sensor nodes. in this paper, we derive a distance function between two psts and propose an algorithm to determine the similarity between them. by the distance between psts, we propose a clustering algorithm to partition objects with similar moving behaviors into groups. furthermore, for each group, one pst is selected to predict movements of objects within one group. experimental results show that our proposed approaches not only effectively reduce the storage cost but also provide good prediction accuracy.
interpretable and reconfigurable clustering of document datasets by deriving word-based rules. clusters of text documents output by clustering algorithms are often hard to interpret. we describe motivating real-world scenarios that necessitate reconfigurability and high interpretability of clusters and outline the problem of generating clusterings with interpretable and reconfigurable cluster models. we develop a clustering algorithm toward the outlined goal of building interpretable and reconfigurable cluster models; it works by generating rules with disjunctions and conditions on the frequencies of words, to decide on the membership of a document to a cluster. each cluster is comprised of precisely the set of documents that satisfy the corresponding rule. we show that our approach outperforms the unsupervised decision tree approach by huge margins. we show that the purity and f-measure losses to achieve interpretability are as little as 5% and 3% respectively using our approach.
measuring system performance and topic discernment using generalized adaptive-weight mean. standard approaches to evaluating and comparing information retrieval systems compute simple averages of performance statistics across individual topics to measure the overall system performance. however, topics vary in their ability to differentiate among systems based on their retrieval performance. at the same time, systems that perform well on discriminative queries demonstrate notable qualities that should be reflected in the systems' evaluation and ranking. this motivated research on alternative performance measures that are sensitive to the discriminative value of topics and the performance consistency of systems. in this paper we provide a mathematical formulation of a performance measure that postulates the dependence between the system and topic characteristics. we propose the generalized adaptive-weight mean (gawm) measure and show how it can be computed as a fixed point of a function for which the brouwer fixed point theorem applies. this guarantees the existence of a scoring scheme that satisfies the starting axioms and can be used for ranking of both systems and topics. we apply our method to trec experiments and compare the gawm with the standard averages used in trec.
agglomerating local patterns hierarchically with alpha. to increase the relevancy of local patterns discovered from noisy relations, it makes sense to formalize error-tolerance. our starting point is to address the limitations of state-of-the-art methods for this purpose. some extractors perform an exhaustive search w.r.t. a declarative specification of error-tolerance. nevertheless, their computational complexity prevents the discovery of large relevant patterns. alpha is a 3-step method that (1) computes complete collections of closed patterns, possibly error-tolerant ones, from arbitrary n-ary relations, (2) enlarges them by hierarchical agglomeration, and (3) selects the relevant agglomerated patterns.
incremental query evaluation for support vector machines. support vector machines (svms) have been widely used in multimedia retrieval to learn a concept in order to find the best matches. in such a svm active learning environment, the system first processes k sampling queries and top-k uncertain queries to select the candidate data items for training. the user's top-k relevant queries are then evaluated to compute the answer. this approach has shown to be effective. however, it suffers from the scalability problem associated with larger database sizes. to address this limitation, we propose an incremental query evaluation technique for these three types of queries. based on the observation that most queries are not revised dramatically during the iterative evaluation, the proposed technique reuses the results of previous queries to reduce the computation cost. furthermore, this technique takes advantage of a tuned index structure to efficiently prune irrelevant data. as a result, only a small portion of the data set needs to be accessed for query processing. this index structure also provides an inexpensive means to process the set of candidates to evaluate the final query result. this technique can work with different kernel functions and kernel parameters. our experimental results indicate that the proposed technique significantly reduces the overall computation cost, and offers a promising solution to the scalability issue.
to divide and conquer search ranking by learning query difficulty. learning to rank plays an important role in information retrieval. in most of the existing solutions for learning to rank, all the queries with their returned search results are learnt and ranked with a single model. in this paper, we demonstrate that it is highly beneficial to divide queries into multiple groups and conquer search ranking based on query difficulty. to this end, we propose a method which first characterizes a query using a variety of features extracted from user search behavior, such as the click entropy, the query reformulation probability. next, a classification model is built on these extracted features to assign a score to represent how difficult a query is. based on this score, our method automatically divides queries into groups, and trains a specific ranking model for each group to conquer search ranking. experimental results on ranksvm and ranknet with a large-scale evaluation dataset show that the proposed method can achieve significant improvement in the task of web search ranking.
improving binary classification on text problems using differential word features. we describe an efficient technique to weigh word-based features in binary classification tasks and show that it significantly improves classification accuracy on a range of problems. the most common text classification approach uses a document's ngrams (words and short phrases) as its features and assigns feature values equal to their frequency or tfidf score relative to the training corpus. our approach uses values computed as the product of an ngram's document frequency and the difference of its inverse document frequencies in the positive and negative training sets. while this technique is remarkably easy to implement, it gives a statistically significant improvement over the standard bag-of-words approaches using support vector machines on a range of classification tasks. our results show that our technique is robust and broadly applicable. we provide an analysis of why the approach works and how it can generalize to other domains and problems.
using domain ontology for semantic web usage mining and next page prediction. this paper proposes the integration of semantic information drawn from a web application's domain knowledge into all phases of the web usage mining process (preprocessing, pattern discovery, and recommendation/prediction). the goal is to have an intelligent semantics-aware web usage mining framework. this is accomplished by using semantic information in the sequential pattern mining algorithm to prune the search space and partially relieve the algorithm from support counting. in addition, semantic information is used in the prediction phase with low order markov models, for less space complexity and accurate prediction, that will help ambiguous predictions problem. experimental results show that semantics-aware sequential pattern mining algorithms can perform 4 times faster than regular non-semantics-aware algorithms with only 26% of the memory requirement.
instance- and bag-level manifold regularization for aggregate outputs classification. aggregate outputs learning differs from the classical supervised learning setting in that, training samples are packed into bags with only the aggregate outputs (labels for classification or real values for regression) known. this setting of the problem is associated with several kinds of application background. we focus on the aggregate outputs classification problem in this paper, and set up a manifold regularization framework to deal with it. the framework can be of both instance level and bag level for different testing goals. we propose four concrete algorithms based on our framework, each of which can cope with both binary and multi-class scenarios. the experimental results on several datasets suggest that our algorithms outperform the state-of-art technique.
probabilistic models of ranking novel documents for faceted topic retrieval. traditional models of information retrieval assume documents are independently relevant. but when the goal is retrieving diverse or novel information about a topic, retrieval models need to capture dependencies between documents. such tasks require alternative evaluation and optimization methods that operate on different types of relevance judgments. we define faceted topic retrieval as a particular novelty-driven task with the goal of finding a set of documents that cover the different facets of an information need. a faceted topic retrieval system must be able to cover as many facets as possible with the smallest number of documents. we introduce two novel models for faceted topic retrieval, one based on pruning a set of retrieved documents and one based on retrieving sets of documents through direct optimization of evaluation measures. we compare the performance of our models to mmr and the probabilistic model due to zhai et al. on a set of 60 topics annotated with facets, showing that our models are competitive.
stereotrust: a group based personalized trust model. trust plays important roles in diverse decentralized environments, including our society at large. computational trust models help to, for instance, guide users' judgements in online auction sites about other users; or determine quality of contributions in web 2.0 sites. most of the existing trust models, however, require historical information about past behavior of a specific agent being evaluated - information that is not always available. in contrast, in real life interactions among users, in order to make the first guess about the trustworthiness of a stranger, we commonly use our "instinct" - essentially stereotypes developed from our past interactions with "similar" people. we propose stereotrust, a computational trust model inspired by real life stereotypes. a user forms stereotypes using her previous transactions with other agents. a stereotype contains certain features of agents and an expected outcome of the transaction. these features can be taken from agents' profile information, or agents' observed behavior in the system. when facing a stranger, the stereotypes matching stranger's profile are aggregated to derive his expected trust. additionally, when some information about stranger's previous transactions is available, stereotrust uses it to refine the stereotype matching. according to our experiments, stereotrust compares favorably with existing trust models that use different kind of information and more complete historical information. moreover, because evaluation is done according to user's personal stereotypes, the system is completely distributed and the result obtained is personalized. stereotrust can be used as a complimentary mechanism to provide the initial trust value for a stranger, especially when there is no trusted, common third parties.
ofcourse: web content discovery, classification and information extraction for online course materials. in this paper we present ofcourse, a vertical search engine for online course materials. these materials have the following characteristics: they are scattered very sparsely in the university web sites; and are generated by the teachers with totally different hmtl templates and layouts. these characteristics impose some challenges for web classification (to identify the course materials) and web information extraction (to extract course metadata, such as course title, time and id) from the identified course homepages. here, we describe our proposed method to tackle these challenges, and the features of this system. ofcourse, containing over 60,000 courses from the top 50 universities in the us, is currently available for public access at http://fusion.hpl.hp.com/ofcourse/.
automatic generation of topic pages using query-based aspect models. we investigate the automatic generation of topic pages as an alternative to the current web search paradigm. we describe a general framework, which combines query log analysis to build aspect models, sentence selection methods for identifying relevant and non-redundant web sentences, and a technique for sentence ordering. we evaluate our approach on biographical topics both automatically and manually, by using wikipedia as reference.
bitmap indexes for relational xml twig query processing. due to an increasing volume of xml data, it is considered prudent to store xml data on an industry-strength database system instead of relying on a domain specific application or a file system. for shredded xml data stored in the relational tables, however, it may not be straightforward to apply existing algorithms for twig query processing, because most of the algorithms require xml data to be accessed in a form of streams of elements grouped by their tags and sorted in a particular order. in order to support xml query processing within the common framework of relational database systems, we first propose several bitmap indexes for supporting holistic twig joins on xml data stored in the relational tables. since bitmap indexes are well supported in most of the commercial and open-source database systems, the proposed bitmap indexes and twig query processing algorithms can be incorporated into the relational query processing framework with more ease. the proposed query processing algorithms are efficient in terms of both time and space, since the compressed bitmap indexes stay compressed during query processing. in addition, we propose a hybrid index which computes twig query solutions with only bit-vectors, without accessing labeled xml elements stored in the relational tables.
effective, design-independent xml keyword search. keyword search techniques that take advantage of xml structure make it very easy for ordinary users to query xml databases, but current approaches to processing these queries rely on intuitively appealing heuristics that are ultimately ad hoc. these approaches often retrieve irrelevant answers, overlook relevant answers, and cannot rank answers appropriately. to address these problems for data-centric xml, we propose coherency ranking (cr), a domain- and database design-independent ranking method for xml keyword queries that is based on an extension of the concept of mutual information. with cr, the results of a keyword query are invariant under schema reorganization. we analyze how previous approaches to xml keyword search approximate cr, and present efficient algorithms to perform cr. our empirical evaluation with 65 user-supplied queries over two real-world xml data sets shows that cr has better precision and recall and provides better ranking than all previous approaches.
product query classification. web query classification is an effective way to understand web user intents, which can further improve web search and online advertising relevance. however, web queries are usually very short which cannot fully reflect their meanings. what is more, it is quite hard to obtain enough training data for training accurate classifiers. therefore, previous work on query classification has focused on two issues. one is how to represent web queries through query expansion. the other is how to increase the amount of training data. in this paper, we took product query classification as an example, which is to classify web queries into a predefined product taxonomy, and systematically studied the impact of query expansion and the size of training data. we proposed two methods of enriching web queries and three approaches of collecting training data. thereafter, we conducted a series of experiments to compare the classification performance of using different combinations of training data and query representations over a real data set. the data set consists of hundreds of thousands queries collected from a popular commercial search engine. from the experiments, we found some interesting observations, which were not discussed before. finally, we proposed an effective and efficient product query classification method based on our observations.
on the feasibility of multi-site web search engines. web search engines are often implemented as centralized systems. designing and implementing a web search engine in a distributed environment is a challenging engineering task that encompasses many interesting research questions. however, distributing a search engine across multiple sites has several advantages, such as utilizing less compute resources and exploiting data locality. in this paper we investigate the cost-effectiveness of building a distributed web search engine. we propose a model for assessing the total cost of a distributed web search engine that includes the computational costs and the communication cost among all distributed sites. we then present a query-processing algorithm that maximizes the amount of queries answered locally, without sacrificing the quality of the results compared to a centralized search engine. we simulate the algorithm on real document collections and query workloads to measure the actual parameters needed for our cost model, and we show that a distributed search engine can be competitive compared to a centralized architecture with respect to real cost.