waashk/dblp huggingface dataset:
incorporating robustness into web ranking evaluation. in many web search engines, a ranking function is selected for deployment mainly by comparing the relevance measurements over candidates. due to the dynamical nature of the web, the ranking features and the query and url distribution on which the ranking functions are built, may change dramatically over time. the actual relevance of the function may degrade, and thus the previous function selection conclusions become invalid. in this work we suggest to select web ranking functions according to both their relevance and robustness to the changes that may lead to relevance degradation over time. we argue that the ranking robustness can be effectively measured by taking into account the ranking score distribution across search results. we then propose two alternatives to the ndcg metric that both incorporate ranking robustness into ranking function evaluation and selection. a machine learning approach is developed to learn the parameters that control the metric sensitivity to score turbulence, from human-judged preference data.
framework for timely and accurate ads on mobile devices. we propose a framework for mobile advertising covering value-added services where an ad-database is maintained on the device and both selection and display are dictated by the device. advantages over existing mobile marketing are that ads are more timely, viable on a variety of use cases, can be both location-sensitive and personalized with minimal privacy concerns, and provide an obvious means for subsidizing users' service costs. we construct a suitable selection algorithm and evaluate its execution, accuracy and scalability. we show that ad-serving can be done under the processing constraints imposed by mobiles, which may lead to improvements in mobile marketing effectiveness.
helping editors choose better seed sets for entity set expansion. sets of named entities are used heavily at commercial search engines such as google, yahoo and bing. acquiring sets of entities typically consists of combining semi-supervised expansion algorithms with manual cleaning of the resulting expanded sets. in this paper, we study the effects of different seed sets in a state-of-the-art semi-supervised expansion system and show a tremendous variation in expansion performance depending on the choice of seeds. we further show that human editors, in general, provide very bad seed sets, which perform well-below the average random seed set. we identify three factors of seed set composition, namely prototypicality, ambiguity and coverage, and we investigate their effects on expansion performance. finally, we propose various automatic systems for improving editor-generated seed sets, which seek to remove ambiguous and other error-prone seed instances. an extensive experimental analysis shows that expansion quality, measured in r-precision, can be improved on average by a maximum of 46% by removing the right seeds from a seed set. our automatic methods outperform the human editors seed sets and on average improve expansion performance by up to 34% over the original seed sets.
semi-supervised learning of semantic classes for query understanding: from the web and for the web. understanding intents from search queries can improve a user's search experience and boost a site's advertising profits. query tagging via statistical sequential labeling models has been shown to perform well, but annotating the training set for supervised learning requires substantial human effort. domain-specific knowledge, such as semantic class lexicons, reduces the amount of needed manual annotations, but much human effort is still required to maintain these as search topics evolve over time. this paper investigates semi-supervised learning algorithms that leverage structured data (html lists) from the web to automatically generate semantic-class lexicons, which are used to improve query tagging performance - even with far less training data. we focus our study on understanding the correct objectives for the semi-supervised lexicon learning algorithms that are crucial for the success of query tagging. prior work on lexicon acquisition has largely focused on the precision of the lexicons, but we show that precision is not important if the lexicons are used for query tagging. a more adequate criterion should emphasize a trade-off between maximizing the recall of semantic class instances in the data, and minimizing the confusability. this ensures that the similar levels of precision and recall are observed on both training and test set, hence prevents over-fitting the lexicon features. experimental results on retail product queries show that enhancing a query tagger with lexicons learned with this objective reduces word level tagging errors by up to 25% compared to the baseline tagger that does not use any lexicon features. in contrast, lexicons obtained through a precision-centric learning algorithm even degrade the performance of a tagger compared to the baseline. furthermore, the proposed method outperforms one in which semantic class lexicons have been extracted from a database.
ossobook: database and knowledgemanagement techniques for archaeozoology. this demo describes the ossobook database system developed for archaeozoology applications providing data storage, data retrieval, and data mining facilities. it shows a case study of integrating state-of-the-art database concepts like intermittently synchronized database system as well as concepts of information retrieval and knowledge representation like similarity search and data mining in order to provide a comprehensive system for an interesting application domain.
smoothing dcg for learning to rank: a novel approach using smoothed hinge functions. discounted cumulative gain (dcg) is widely used for evaluating ranking functions. it is therefore natural to learn a ranking function that directly optimizes dcg. however, dcg is non-smooth, rendering gradient-based optimization algorithms inapplicable. to remedy this, smoothed versions of dcg have been proposed but with only partial success. in this paper, we first present analysis that shows it is ineffective using the gradient of the smoothed dcg to drive the optimization algorithm. we then propose a novel approach, shf-sdcg, for smoothing dcg by using smoothed hinge functions (shf). it has the advantage of seamlessly transition from driving the optimization mimicking pairwise learning when the ranking function does not fit the data well, to driving the optimization using dcg when the ranking function becomes more accurate. shf-sdcg is then extended to reg-shf-sdcg, an algorithm which gradually transits from pointwise and pairwise to listwise learning. finally experimental results are provided to validate the effectiveness of shf-sdcg and reg-shf-sdcg.
enhancing recommender systems under volatile userinterest drifts. this paper presents a systematic study of how to enhance recommender systems under volatile user interest drifts. a key development challenge along this line is how to track user interests dynamically. to this end, we first define four types of interest patterns to understand users' rating behaviors and analyze the properties of these patterns. we also propose a rating graph and rating chain based approach for detecting these interest patterns. for each users' rating series, a rating graph and a rating chain are constructed based on the similarities between rated items. the type of a given user's interest pattern is identified through the density of the corresponding rating graph and the continuity of the corresponding rating chain. in addition, we propose a general algorithm framework for improving recommender systems by exploiting these identified patterns. finally, experimental results on a real-world data set show that the proposed rating graph based approach is effective for detecting user interest patterns, which in turn help to improve the performance of recommender systems.
a social recommendation framework based on multi-scale continuous conditional random fields. this paper addresses the issue of social recommendation based on collaborative filtering (cf) algorithms. social recommendation emphasizes utilizing various attributes information and relations in social networks to assist recommender systems. although recommendation techniques have obtained distinct developments over the decades, traditional cf algorithms still have these following two limitations: (1) relational dependency within predictions, an important factor especially when the data is sparse, is not being utilized effectively; and (2) straightforward methods for combining features like linear integration suffer from high computing complexity in learning the weights by enumerating the whole value space, making it difficult to combine various information into an unified approach. in this paper, we propose a novel model, multi-scale continuous conditional random fields (mccrf), as a framework to solve above problems for social recommendations. in mccrf, relational dependency within predictions is modeled by the markov property, thus predictions are generated simultaneously and can help each other. this strategy has never been employed previously. besides, diverse information and relations in social network can be modeled by state and edge feature functions in mccrf, whose weights can be optimized globally. thus both problems can be solved under this framework. in addition, we propose to utilize markov chain monte carlo (mcmc) estimation methods to solve the difficulties in training and inference processes of mccrf. experimental results conducted on two real world data have demonstrated that our approach outperforms traditional cf algorithms. additional experiments also show the improvements from the two factors of relational dependency and feature combination, respectively.
aging effects on query flow graphs for query suggestion. world wide web content continuously grows in size and importance. furthermore, users ask web search engines to satisfy increasingly disparate information needs. new techniques and tools are constantly developed aimed at assisting users in the interaction with the web search engine. query recommender systems suggesting interesting queries to users are an example of such tools. most query recommendation techniques are based on the knowledge of the behaviors of past users of the search engine recorded in query logs. a recent query-log mining approach for query recommendation is based on query flow graphs (qfg). in this paper we propose an evaluation of the effects of time on this query recommendation model. as users interests change over time, the knowledge extracted from query logs may suffer an aging effect as new interesting topics appear. in order to validate experimentally this hypothesis, we build different query flow graphs from the queries belonging to a large query log of a real-world search engine. each query flow graph is built on distinct query log segments. then, we generate recommendations on different sets of queries. results are assessed both by means of human judgments and by using an automatic evaluator showing that the models inexorably age.
independent informative subgraph mining for graph information retrieval. in order to enable scalable querying of graph databases, intelligent selection of subgraphs to index is essential. an improved index can reduce response times for graph queries significantly. for a given subgraph query, graph candidates that may contain the subgraph are retrieved using the graph index and subgraph isomorphism tests are performed to prune out unsatisfied graphs. however, since the space of all possible subgraphs of the whole set of graphs is prohibitively large, feature selection is required to identify a good subset of subgraph features for indexing. thus, one of the key issues is: given the set of all possible subgraphs of the graph set, which subset of features is the optimal such that the algorithm retrieves the smallest set of candidate graphs and reduces the number of subgraph isomorphism tests? we introduce a graph search method for subgraph queries based on subgraph frequencies. then, we propose several novel feature selection criteria, max-precision, max-irredundant-information, and max-information-min-redundancy, based on mutual information. finally we show theoretically and empirically that our proposed methods retrieve a smaller candidate set than previous methods. for example, using the same number of features, our method improve the precision for the query candidate set by 4%-13% in comparison to previous methods. as a result the response time of subgraph queries also is improved correspondingly.
vetting the links of the web. many web links mislead human surfers and automated crawlers because they point to changed content, out-of-date information, or invalid urls. it is a particular problem for large, well-known directories such as the dmoz open directory project, which maintains links to representative and authoritative external web pages within their various topics. therefore, such sites involve many editors to manually revisit and revise links that have become out-of-date. to remedy this situation, we propose the novel web mining task of identifying outdated links on the web. we build a general classification model, primarily using local and global temporal features extracted from historical content, topic, link and time-focused changes over time. we evaluate our system via five-fold cross-validation on more than fifteen thousand odp external links selected from thirteen top-level categories. our system can predict the actions of odp editors more than 75% of the time. our models and predictions could be useful for various applications that depend on analysis of web links, including ranking and crawling.
refmed: relevance feedback retrieval system fo pubmed. finding related articles from the pubmed (a large biomedical literature repository) is challenging because it is hard to express the user's specific relevance in the given query interface and a keyword query typically retrieves many results. biomedical researchers spend a critical amount of time (e.g., often more than several days) in the literature search process. this paper proposes refmed, a novel search system for pubmed, which supports relevance ranking by enabling relevance feedback on pubmed. refmed first returns initial result documents for a user's keyword query as in pubmed. the user then makes relevance judgments on some of the resultant documents while browsing them. once the user "pushes the feedback", the system induces a relevance function using ranksvm and ranks the results according to the function. to realize the ad-hoc relevance retrieval on pubmed, refmed "tightly" integrates ranksvm within rdbms and runs the rank learning and process on the fly with a response time of a few minutes.our qualitative experiments with biomedical researchers show that refmed substantially reduces the amount of effort required to search related pubmed articles. refmed is accessible at "http://dm.postech.ac.kr/refmed".
characterizing commercial intent. understanding the intent underlying user's queries may help personalize search results and therefore improve user satisfaction. we develop a methodology for using the content of search engine result pages (serps) along with the information obtained from query strings to study characteristics of query intent, with a particular focus on sponsored search. this work represents an initial step towards the development and evaluation of an ontology for commercial search, considering queries that reference specific products, brands and retailers. the characteristics of query categories are studied with respect to aggregated user's clickthrough behavior on advertising links. we present a model for clickthrough behavior that considers the influence of such factors as the location of ads and the rank of ads, along with query category. we evaluate our work using a large corpus of clickthrough data obtained from a major commercial search engine. our findings suggest that query based features, along with the content of serps, are effective in detecting query intent. the clickthrough behavior is found to be consistent with the classification for the general categories of query intent, while for product, brand and retailer categories, all is true to a lesser extent.
acronym extraction and disambiguation in large-scale organizational web pages. in this paper, we focus on the automatic extraction and disambiguation of acronyms in large-scale organizational web pages, which is important but difficult due to the diversity of acronyms and the scale of organizational web pages. we propose two novel algorithms to address the key problems in acronym extraction and disambiguation: (1) an unsupervised ranking algorithm to automatically filter out the incorrect acronym-expansion pairs. different from the existing approaches, our method does not require any hand-crafted rules; (2) a graph-based algorithm to disambiguate ambiguous acronyms, which leverages the hyperlinks of pages to facilitate the acronym disambiguation. we evaluate the proposed approaches using two large-scale, real-world datasets in two different domains. our experimental results show that our approach is domain independent, and achieves higher precision and recall than the existing methods.
imecho: an associative memory based desktop search system. traditional desktop search engines only support keyword based search that needs exact keyword matching to find resources. however, users generally have a vague picture of what is stored but forget the exact location and keywords of the resource. according to observations of human associative memory, people tend to remember things from some memory fragments in their brains and these memory fragments are connected by memory cues of user activity context. we developed imecho (my memory echo), an associative memory based desktop search system, which exploits such associations and contexts to enhance traditional desktop search. desktop resources are connected with semantic links mined from explicit and implicit user activities according to specific access patterns. using these semantic links, associations among memory fragments can be built or rebuilt in a user's brain during a search. moreover, our personalized ranking scheme uses these links together with a user's personal preferences to rank results by both relevance and importance to the user. in addition, the system provides a faceted search feature and association graph navigation to help users refine and associate search results generated by full-text keyword search. our experiments investigating precision and recall quality of imecho prototype show that the association-based search system is superior to the traditional keyword search in personal search engines since it is closer to the way that human associative memory works.
nonlinear static-rank computation. mainstream link-based static-rank algorithms (e.g. pagerank and its variants) express the importance of a page as the linear combination of its in-links and compute page importance scores by solving a linear system in an iterative way. such linear algorithms, however, may give apparently unreasonable static-rank results for some link structures. in this paper, we examine the static-rank computation problem from the viewpoint of evidence combination and build a probabilistic model for it. based on the model, we argue that a nonlinear formula should be adopted, due to the correlation or dependence between links. we focus on examining some simple formulas which only consider the correlation between links in the same domain. experiments conducted on 100 million web pages (with multiple static-rank quality evaluation metrics) show that higher quality static-rank could be yielded by the new nonlinear algorithms. the convergence of the new algorithms is also proved in this paper by nonlinear functional analysis.
a machine learning approach for improved bm25 retrieval. despite the widespread use of bm25, there have been few studies examining its effectiveness on a document description over single and multiple field combinations. we determine the effectiveness of bm25 on various document fields. we find that bm25 models relevance on popularity fields such as anchor text and query click information no better than a linear function of the field attributes. we also find query click information to be the single most important field for retrieval. in response, we develop a machine learning approach to bm25-style retrieval that learns, using lambdarank, from the input attributes of bm25. our model significantly improves retrieval effectiveness over bm25 and bm25f. our data-driven approach is fast, effective, avoids the problem of parameter tuning, and can directly optimize for several common information retrieval measures. we demonstrate the advantages of our model on a very large real-world web data collection.
probabilistic moving range query over rfid spatio-temporal data streams. moving range query over rfid data streams is one of the most important spatio-temporal queries to support valuable information analysis. however, the location uncertainty challenges the query strategy. in this paper, we propose a probability evaluation model in the rfid-enabled monitoring environments and discuss the query optimization techniques under the scenarios of continuous moving range query, which can also be applied into more situations. the extensive experimental evaluation verifies the efficiency and effectiveness of our proposed model and methods.
similarity-aware indexing for real-time entity resolution. entity resolution, also known as data matching or record linkage, is the task of identifying and matching records from several databases that refer to the same entities. traditionally, entity resolution has been applied in batch-mode and on static databases. however, many organisations are increasingly faced with the challenge of having large databases containing entities that need to be matched in real-time with a stream of query records also containing entities, such that the best matching records are retrieved. example applications include online law enforcement and national security databases, public health surveillance and emergency response systems, financial verification systems, online retail stores, egovernment services, and digital libraries. a novel inverted index based approach for real-time entity resolution is presented in this paper. at build time, similarities between attribute values are computed and stored to support the fast matching of records at query time. the presented approach differs from other approaches to approximate query matching in that it allows any similarity comparison function, and any 'blocking' (encoding) function, both possibly domain specific, to be incorporated. experimental results on a real-world database indicate that the total size of all data structures of this novel index approach grows sub-linearly with the size of the database, and that it allows matching of query records in sub-second time, more than two orders of magnitude faster than a traditional entity resolution index approach. the interested reader is referred to the longer version of this paper [5].
walking in the crowd: anonymizing trajectory data for pattern analysis. recently, trajectory data mining has received a lot of attention in both the industry and the academic research. in this paper, we study the privacy threats in trajectory data publishing and show that traditional anonymization methods are not applicable for trajectory data due to its challenging properties: high-dimensional, sparse, and sequential. our primary contributions are (1) to propose a new privacy model called lkc-privacy that overcomes these challenges, and (2) to develop an efficient anonymization algorithm to achieve lkc-privacy while preserving the information utility for trajectory pattern mining.