waashk/dblp huggingface dataset:
anticipating annotations and emerging trends in biomedical literature. the biojournalmonitor is a decision support system for the analysis of trends and topics in the biomedical literature. its main goal is to identify potential diagnostic and therapeutic biomarkers for specific diseases. several data sources are continuously integrated to provide the user with up-to-date information on current research in this field. state-of-the-art text mining technologies are deployed to provide added value on top of the original content, including named entity detection, relation extraction, classification, clustering, ranking, summarization, and visualization. we present two novel technologies that are related to the analysis of temporal dynamics of text archives and associated ontologies. currently, the mesh ontology is used to annotate the scientific articles entering the pubmed database with medical terms. both the maintenance of the ontology as well as the annotation of new articles is performed largely manually. we describe how probabilistic topic models can be used to annotate recent articles with the most likely mesh terms. this provides our users with a competitive advantage because, when searching for mesh terms, articles are found long before they are manually annotated. we further present a study on how to predict the inclusion of new terms in the mesh ontology. the results suggest that early prediction of emerging trends is possible. the trend ranking functions are deployed in our system to enable interactive searches for the hottest new trends relating to a disease.
partial least squares regression for graph mining. attributed graphs are increasingly more common in many application domains such as chemistry, biology and text processing. a central issue in graph mining is how to collect informative subgraph patterns for a given learning task. we propose an iterative mining method based on partial least squares regression (pls). to apply pls to graph data, a sparse version of pls is developed first and then it is combined with a weighted pattern mining algorithm. the mining algorithm is iteratively called with different weight vectors, creating one latent component per one mining call. our method, graph pls, is efficient and easy to implement, because the weight vector is updated with elementary matrix calculations. in experiments, our graph pls algorithm showed competitive prediction accuracies in many chemical datasets and its efficiency was significantly superior to graph boosting (gboost) and the naive method based on frequent graph mining.
a sequential dual method for large scale multi-class linear svms. efficient training of direct multi-class formulations of linear support vector machines is very useful in applications such as text classification with a huge number examples as well as features. this paper presents a fast dual method for this training. the main idea is to sequentially traverse through the training set and optimize the dual variables associated with one example at a time. the speed of training is enhanced further by shrinking and cooling heuristics. experiments indicate that our method is much faster than state of the art solvers such as bundle, cutting plane and exponentiated gradient methods.
semi-supervised approach to rapid and reliable labeling of large data sets. in this paper, we propose a method, where the labeling of the data set is carried out in a semi-supervised manner with user-specified guarantees about the quality of the labeling. in our scheme, we assume that for each class, we have some heuristics available, each of which can identify instances of one particular class. the heuristics are assumed to have reasonable performance but they do not need to cover all instances of the class nor do they need to be perfectly reliable. we further assume that we have an infallible expert, who is willing to manually label a few instances. the aim of the algorithm is to exploit the cluster structure of the problem, the predictions by the imperfect heuristics and the limited perfect labels provided by the expert to classify (label) the instances of the data set with guaranteed precision (specificed by the user) with regards to each class. the specified precision is not always attainable, so the algorithm is allowed to classify some instances as dontknow. the algorithm is evaluated by the number of instances labeled by the expert, the number of dontknow instances (global coverage) and the achieved quality of the labeling. on the kdd cup network intrusion data set containing 500,000 instances, we managed to label 96.6% of the instances while guaranteeing a nominal precision of 90% (with 95% confidence) by having the expert label 630 instances; and by having the expert label 1200 instances, we managed to guarantee 95% nominal precision while labeling 96.4% of the data. we also provide a case study of applying our scheme to label the network traffic collected at a large campus network.
social networks: looking ahead. by now, online social networks have become an indispensable part of both online and offline lives of human beings. a large fraction of time spent online by a user is directly influence by the social networks to which he/she belongs. this calls for a deeper examination of social networks as large-scale dynamic objects that foster efficient person-person interaction. the goal of our panel is to discuss social networks from various research angles. in particular, we plan to focus on the following broad research-related topics: large scale data mining, algorithmic questions, sociological aspects, privacy, web search, etc. we will also discuss the business and societal impacts of social networks. each of these topics has generated a lot of research in recent years and while taking stock of what has been done, we will also be discussing the directions in which these topics are headed, from both science and society points of view. our panel will consist of eminent researchers, who have worked/been working on an eclectic and diverse mix of problems in social networks
identifying domain expertise of developers from source code. we are interested in identifying the domain expertise of developers of a software system. a developer gains expertise on the code base as well as the domain of the software system he/she develops. this information forms a useful input in allocating software implementation tasks to developers. domain concepts represented by the system are discovered by taking into account the linguistic information available in the source code. the vocabulary contained in source code as identifiers such as class, method, variable names and comments are extracted. concepts present in the code base are identified and grouped based on a well known text processing hypothesis - words are similar to the extent to which they share similar words. the developer's association with the source code and the concepts it represents is arrived at using the version repository information. in this line, the analysis first derives documents from source code by discarding all the programming language constructs. kmeans clustering is further used to cluster documents and extract closely related concepts. the key concepts present in the documents authored by the developer determine his/her domain expertise. to validate our approach we apply it on large software systems, two of which are presented in detail in this paper.
identifying authoritative actors in question-answering forums: the case of yahoo! answers. we consider the problem of identifying authoritative users in yahoo! answers. a common approach is to use link analysis techniques in order to provide a ranked list of users based on their degree of authority. a major problem for such an approach is determining how many users should be chosen as authoritative from a ranked list. to address this problem, we propose a method for automatic identification of authoritative actors. in our approach, we propose to model the authority scores of users as a mixture of gamma distributions. the number of components in the mixture is estimated by the bayesian information criterion (bic) while the parameters of each component are estimated using the expectation-maximization (em) algorithm. this method allows us to automatically discriminate between authoritative and non-authoritative users. the suitability of our proposal is demonstrated in an empirical study using datasets from yahoo! answers.
de-duping urls via rewrite rules. a large fraction of the urls on the web contain duplicate (or near-duplicate) content. de-duping urls is an extremely important problem for search engines, since all the principal functions of a search engine, including crawling, indexing, ranking, and presentation, are adversely impacted by the presence of duplicate urls. traditionally, the de-duping problem has been addressed by fetching and examining the content of the url; our approach here is different. given a set of urls partitioned into equivalence classes based on the content (urls in the same equivalence class have similar content), we address the problem of mining this set and learning url rewrite rules that transform all urls of an equivalence class to the same canonical form. these rewrite rules can then be applied to eliminate duplicates among urls that are encountered for the first time during crawling, even without fetching their content. in order to express such transformation rules, we propose a simple framework that is general enough to capture the most common url rewrite patterns occurring on the web; in particular, it encapsulates the dust (different urls with similar text) framework [5]. we provide an efficient algorithm for mining and learning url rewrite rules and show that under mild assumptions, it is complete, i.e., our algorithm learns every url rewrite rule that is correct, for an appropriate notion of correctness. we demonstrate the expressiveness of our framework and the effectiveness of our algorithm by performing a variety of extensive large-scale experiments.
angle-based outlier detection in high-dimensional data. detecting outliers in a large set of data objects is a major data mining task aiming at finding different mechanisms responsible for different groups of objects in a data set. all existing approaches, however, are based on an assessment of distances (sometimes indirectly by assuming certain distributions) in the full-dimensional euclidean data space. in high-dimensional data, these approaches are bound to deteriorate due to the notorious "curse of dimensionality". in this paper, we propose a novel approach named abod (angle-based outlier detection) and some variants assessing the variance in the angles between the difference vectors of a point to the other points. this way, the effects of the "curse of dimensionality" are alleviated compared to purely distance-based approaches. a main advantage of our new approach is that our method does not rely on any parameter selection influencing the quality of the achieved ranking. in a thorough experimental evaluation, we compare abod to the well-established distance-based method lof for various artificial and a real world data set and show abod to perform especially well on high-dimensional data.
constraint programming for itemset mining. the relationship between constraint-based mining and constraint programming is explored by showing how the typical constraints used in pattern mining can be formulated for use in constraint programming environments. the resulting framework is surprisingly flexible and allows us to combine a wide range of mining constraints in different ways. we implement this approach in off-the-shelf constraint programming systems and evaluate it empirically. the results show that the approach is not only very expressive, but also works well on complex benchmark problems.
can complex network metrics predict the behavior of nba teams? the united states national basketball association (nba) is one of the most popular sports league in the world and is well known for moving a millionary betting market that uses the countless statistical data generated after each game to feed the wagers. this leads to the existence of a rich historical database that motivates us to discover implicit knowledge in it. in this paper, we use complex network statistics to analyze the nba database in order to create models to represent the behavior of teams in the nba. results of complex network-based models are compared with box score statistics, such as points, rebounds and assists per game. we show the box score statistics play a significant role for only a small fraction of the players in the league. we then propose new models for predicting a team success based on complex network metrics, such as clustering coefficient and node degree. complex network-based models present good results when compared to box score statistics, which underscore the importance of capturing network relationships in a community such as the nba.
identifying biologically relevant genes via multiple heterogeneous data sources. selection of genes that are differentially expressed and critical to a particular biological process has been a major challenge in post-array analysis. recent development in bioinformatics has made various data sources available such as mrna and mirna expression profiles, biological pathway and gene annotation, etc. efficient and effective integration of multiple data sources helps enrich our knowledge about the involved samples and genes for selecting genes bearing significant biological relevance. in this work, we studied a novel problem of multi-source gene selection: given multiple heterogeneous data sources (or data sets), select genes from expression profiles by integrating information from various data sources. we investigated how to effectively employ information contained in multiple data sources to extract an intrinsic global geometric pattern and use it in covariance analysis for gene selection. we designed and conducted experiments to systematically compare the proposed approach with representative methods in terms of statistical and biological significance, and showed the efficacy and potential of the proposed approach with promising findings.
knowledge discovery of semantic relationships between words using nonparametric bayesian graph model. we developed a model based on nonparametric bayesian modeling for automatic discovery of semantic relationships between words taken from a corpus. it is aimed at discovering semantic knowledge about words in particular domains, which has become increasingly important with the growing use of text mining, information retrieval, and speech recognition. the subject-predicate structure is taken as a syntactic structure with the noun as the subject and the verb as the predicate. this structure is regarded as a graph structure. the generation of this graph can be modeled using the hierarchical dirichlet process and the pitman-yor process. the probabilistic generative model we developed for this graph structure consists of subject-predicate structures extracted from a corpus. evaluation of this model by measuring the performance of graph clustering based on wordnet similarities demonstrated that it outperforms other baseline models.
entity categorization over large document collections. extracting entities (such as people, movies) from documents and identifying the categories (such as painter, writer) they belong to enable structured querying and data analysis over unstructured document collections. in this paper, we focus on the problem of categorizing extracted entities. most prior approaches developed for this task only analyzed the local document context within which entities occur. in this paper, we significantly improve the accuracy of entity categorization by (i) considering an entity's context across multiple documents containing it, and (ii) exploiting existing large lists of related entities (e.g., lists of actors, directors, books). these approaches introduce computational challenges because (a) the context of entities has to be aggregated across several documents and (b) the lists of related entities may be very large. we develop techniques to address these challenges. we present a thorough experimental study on real data sets that demonstrates the increase in accuracy and the scalability of our approaches.
efficient computation of personal aggregate queries on blogs. there is an exploding amount of user-generated content on theweb due to the emergence of "web 2.0" services, such as blogger,myspace, flickr, and del.icio.us. the participation of a large number of users in sharing their opinion on the web has inspired researchers to build an effective "information filter" by aggregating these independent opinions. however, given the diverse groups of users on the web nowadays, the global aggregation of the information may not be of much interest to different groups of users. in this paper, we explore the possibility of computing personalized aggregation over the opinions expressed on the web based on a user's indication of trust over the information sources. the hope is that by employing such "personalized" aggregation, we can make the recommendation more likely to be interesting to the users. we address the challenging scalability issues by proposing an efficient method, that utilizes two core techniques: non-negative matrix factorization and threshold algorithm, to compute personalized aggregations when there are potentially millions of users and millions of sources within a system. we show that, through experiments on real-life dataset, our personalized aggregation approach indeed makes a significant difference in the items that are recommended and it reduces the query computational cost significantly, often more than 75%, while the result of personalized aggregation is kept accurate enough.
the persuasive phase of visualization. research in visualization often revolves around visualizing information. however, visualization is a process that extends over time from initial exploration to hypothesis confirmation, and even to result presentation. it is rare that the final phases of visualization are solely about information. in this paper we present a more biased kind of visualization, in which there is a message or set of assumptions behind the presentation that is of interest to both the presenter and the viewer, and emphasizes points that the presenter wants to convey to the viewer. this kind of persuasive visualization -- presenting data in a way that emphasizes a point or message -- is not only common in visualization, but also often expected by the viewer. persuasive visualization is implicit in the deliberate emphasis on interestingness and also in the deliberate use of graphical elements that are processed preattentively by the human visual system, which automatically groups these elements and guiding attention so that they "stand out". we discuss how these ideas have been implemented in the morpherspective system for automated generation of information graphics.
bypass rates: reducing query abandonment using negative inferences. we introduce a new approach to analyzing click logs by examining both the documents that are clicked and those that are bypassed-documents returned higher in the ordering of the search results but skipped by the user. this approach complements the popular click-through rate analysis, and helps to draw negative inferences in the click logs. we formulate a natural objective that finds sets of results that are unlikely to be collectively bypassed by a typical user. this is closely related to the problem of reducing query abandonment. we analyze a greedy approach to optimizing this objective, and establish theoretical guarantees of its performance. we evaluate our approach on a large set of queries, and demonstrate that it compares favorably to the maximal marginal relevance approach on a number of metrics including mean average precision and mean reciprocal rank.
land cover change detection: a case study. the study of land cover change is an important problem in the earth science domain because of its impacts on local climate, radiation balance, biogeochemistry, hydrology, and the diversity and abundance of terrestrial species. most well-known change detection techniques from statistics, signal processing and control theory are not well-suited for the massive high-dimensional spatio-temporal data sets from earth science due to limitations such as high computational complexity and the inability to take advantage of seasonality and spatio-temporal autocorrelation inherent in earth science data. in our work, we seek to address these challenges with new change detection techniques that are based on data mining approaches. specifically, in this paper we have performed a case study for a new change detection technique for the land cover change detection problem. we study land cover change in the state of california, focusing on the san francisco bay area and perform an extended study on the entire state. we also perform a comparative evaluation on forests in the entire state. these results demonstrate the utility of data mining techniques for the land cover change detection problem.
the cost of privacy: destruction of data-mining utility in anonymized data publishing. re-identification is a major privacy threat to public datasets containing individual records. many privacy protection algorithms rely on generalization and suppression of "quasi-identifier" attributes such as zip code and birthdate. their objective is usually syntactic sanitization: for example, k-anonymity requires that each "quasi-identifier" tuple appear in at least k records, while l-diversity requires that the distribution of sensitive attributes for each quasi-identifier have high entropy. the utility of sanitized data is also measured syntactically, by the number of generalization steps applied or the number of records with the same quasi-identifier. in this paper, we ask whether generalization and suppression of quasi-identifiers offer any benefits over trivial sanitization which simply separates quasi-identifiers from sensitive attributes. previous work showed that k-anonymous databases can be useful for data mining, but k-anonymization does not guarantee any privacy. by contrast, we measure the tradeoff between privacy (how much can the adversary learn from the sanitized records?) and utility, measured as accuracy of data-mining algorithms executed on the same sanitized records. for our experimental evaluation, we use the same datasets from the uci machine learning repository as were used in previous research on generalization and suppression. our results demonstrate that even modest privacy gains require almost complete destruction of the data-mining utility. in most cases, trivial sanitization provides equivalent utility and better privacy than k-anonymity, l-diversity, and similar methods based on generalization and suppression.