waashk/dblp huggingface dataset:
a novel method for stemmer generation based on hidden markov models. in this paper, we present a method based on hidden markov models (hmms) to generate statistical stemmers. using a list of words as training set, the method estimates the hmm parameters which are used to calculate the most probable stem for an arbitrary word. stemming is performed by computing the most probable path, through the hmm states, corresponding to the input word. linguistic knowledge or a training set of manually stemmed words are not required. we describe the method and the results of the experiments carried out using standard test collections for five different languages.
unified filtering by combining collaborative filtering and content-based filtering via mixture model and exponential model. collaborative filtering and content-based filtering are two types of information filtering techniques. combining these two techniques can improve the recommendation effectiveness. the main problem with previous research is that the content information and the rating information are not combined in an integrated way. this paper presents a unified probabilistic framework that allows the mutual interaction between these two types of information. experiments have shown that the new unified filtering algorithm outperforms a pure collaborative filtering approach, a pure content-based filtering approach and another unified filtering algorithm.
evaluation of a mca-based approach to organize data cubes. in the olap context, exploration of huge and sparse data cubes is a tedious task that does not always lead to efficient results. we propose to use a multiple correspondence analysis (mca) in order to enhance data cube representations and make them more suitable for visualization and thus, easier to analyze. we also provide an original quality criterion to measure the relevance of the obtained data representations. experimental results we led on real data samples have shown the interest and the efficiency of our approach.
ontology-driven peer profiling in peer-to-peer enabled semantic web. peer-to-peer (p2p) systems and semantic web are two novel technologies that face a lot of shortcomings if considered as isolated paradigms. we present an approach that utilizes ontologies to set up a peer profile containing all the data, necessary for peer-to-peer interoperability. using this profile can help eliminate some major issues persistent in current p2p networks, such as security, resource aggregation, group management. we also consider applications of peer profiling for semantic web built on p2p networks, such as an improved semantic search for resources, not explicitly published on the web, but available in a p2p system. we develop the ontology-based peer profile in rdf format and demonstrate its manifold benefits for peer communication and knowledge discovery in both p2p networks and semantic web.
similarity measures for tracking information flow. text similarity spans a spectrum, with broad topical similarity near one extreme and document identity at the other. intermediate levels of similarity -- resulting from summarization, paraphrasing, copying, and stronger forms of topical relevance -- are useful for applications such as information flow analysis and question-answering tasks. in this paper, we explore mechanisms for measuring such intermediate kinds of similarity, focusing on the task of identifying where a particular piece of information originated. we consider both sentence-to-sentence and document-to-document comparison, and have incorporated these algorithms into <small>recap</small>, a prototype information flow analysis tool. our experimental results with <small>recap</small> indicate that new mechanisms such as those we propose are likely to be more appropriate than existing methods for identifying the intermediate forms of similarity.
incremental and interactive sequence mining. the discovery of frequent sequences in temporal databases is an important data mining problem. most current work assumes that the database is static, and a database update requires rediscovering all the patterns by scanning the entire old and new database. in this paper, we propose novel techniques for maintaining sequences in the presence of a) database updates, and b) user interaction (e.g. modifying mining parameters). this is a very challenging task, since such updates can invalidate existing sequences or introduce new ones. in both the above scenarios, we avoid re-executing the algorithm on the entire dataset, thereby reducing execution time. experimental results confirm that our approach results in execution time improvements of up to several orders of magnitude in practice.
document clustering using character n-grams: a comparative evaluation with term-based and word-based clustering. we propose a novel method for document clustering using character n-grams. in the traditional vector-space model, the documents are represented as vectors, in which each dimension corresponds to a word. we propose a document representation based on the most frequent character n-grams, with window size of up to 10 characters. we derive a new distance measure, which produces uniformly better results when compared to the word-based and term-based methods. the result becomes more significant in the light of the robustness of the n-gram method with no language-dependent preprocessing. experiments on the performance of a clustering algorithm on a variety of test document corpora demonstrate that the n-gram representation with n=3 outperforms both word and term representations. the comparison between word and term representations depends on the data set and the selected dimensionality.
using rankboost to compare retrieval systems. this paper presents a new pooling method for constructing the assessment sets used in the evaluation of retrieval systems. our proposal is based on rankboost, a machine learning voting algorithm. it leads to smaller pools than classical pooling and thus reduces the manual assessment workload for building test collections. experimental results obtained on an xml document collection demonstrate the effectiveness of the approach according to different evaluation criteria.
score region algebra: building a transparent xml-r database. a unified database framework that will enable better comprehension of ranked xml retrieval is still a challenge in the xml database field. we propose a logical algebra, named score region algebra, that enables transparent specification of information retrieval (ir) models for xml databases. the transparency is achieved by a possibility to instantiate various retrieval models, using abstract score functions within algebra operators, while logical query plan and operator definitions remain unchanged. our algebra operators model three important aspects of xml retrieval: element relevance score computation, element score propagation, and element score combination. to illustrate the usefulness of our algebra we instantiate four different, well known ir scoring models, and combine them with different score propagation and combination functions. we implemented the algebra operators in a prototype system on top of a low-level database kernel. the evaluation of the system is performed on a collection of ieee articles in xml format provided by inex. we argue that state of the art xml ir models can be transparently implemented using our score region algebra framework on top of any low-level physical database engine or existing rdbms, allowing a more systematic investigation of retrieval model behavior.
acquisition of categorized named entities for web search. the recognition of names and their associated categories within unstructured text traditionally relies on semantic lexicons and gazetteers. the amount of effort required to assemble large lexicons confines the recognition to either a limited domain (e.g., <i>medical imaging</i>), or a small set of pre-defined, broader categories of interest (e.g., <i>persons</i>, <i>countries</i>, <i>organizations</i>, <i>products</i>). this constitutes a serious limitation in an information seeking context. in this case, the categories of potential interest to users are more diverse (<i>universities</i>, <i>agencies</i>, <i>retailers</i>, <i>celebrities</i>), often refined (e.g., <i>slr digital cameras</i>, <i>programming languages</i>, <i>multinational oil companies</i>), and usually overlapping (e.g., the same entity may be concurrently a <i>brand name</i>, a <i>technology company</i>, and an <i>industry leader</i>). we present a lightly supervised method for acquiring named entities in arbitrary categories. the method applies lightweight lexico-syntactic extraction patterns to the unstructured text of web documents. the method is a departure from traditional approaches to named entity recognition in that: 1) it does not require any start-up seed names or training; 2) it does not encode any domain knowledge in its extraction patterns; 3) it is only lightly supervised, and data-driven; 4) it does not impose any a-priori restriction on the categories of extracted names. we illustrate applications of the method in web search, and describe experiments on 500 million web documents and news articles.
automatic analysis of call-center conversations. we describe a system for automating call-center analysis and monitoring. our system integrates transcription of incoming calls with analysis of their content; for the analysis, we introduce a novel method of estimating the domain-specific importance of conversation fragments, based on divergence of corpus statistics. combining this method with information retrieval approaches, we provide knowledge-mining tools both for the call-center agents and for administrators of the center.
mining sequential patterns with constraints in large databases. constraints are essential for many sequential pattern mining applications. however, there is no systematic study on constraint-based sequential pattern mining. in this paper, we investigate this issue and point out that the framework developed for constrained frequent-pattern mining does not fit our missions well. an extended framework is developed based on a sequential pattern growth methodology. our study shows that constraints can be effectively and efficiently pushed deep into sequential pattern mining under this new framework. moreover, this framework can be extended to constraint-based structured pattern mining as well.
similarity among melodies for music information retrieval. here we discuss how to look for similar melody in music databases by giving monophonic melody in sheet. in this work, we utilize text expression (or sheet music) to describe music and introduce pitch spectrum of melodies. by this feature, we concisely distinguish music from tempo, transposition or other arbitrary expressions. we show the usefulness by experimental results.
access control for xml: a dynamic query rewriting approach. being able to express and enforce role-based access control on xml data is a critical component of xml data management. however, given the semi-structured nature of xml, this is non-trivial, as access control can be applied on the values of nodes as well as on the structural relationship between nodes. in this context, we adopt and extend a graph editing language for specifying role-based access constraints in the form of security views. a security annotated schema (sas) is proposed as the internal representation for the security views and can be automatically constructed from the original schema and the security view specification. to enforce the access constraints on user queries, we propose secure query rewrite (sqr) -- a set of rules that can be used to rewrite a user xpath query on the security view into an equivalent xquery expression against the original data, with the guarantee that the users only see information in the view but not any data that was blocked. experimental evaluation demonstrates the efficiency and the expressiveness of our approach.
irregularity in multi-dimensional space-filling curves with applications in multimedia databases. a space-filling curve is a way of mapping the multi-dimensional space into the one-dimensional space. it acts like a thread that passes through every cell element (or pixel) in the n-dimensional space so that every cell is visited at least once. thus, a space-filling curve imposes a linear order of the cells in the n-dimensional space. there are numerous kinds of space-filling curves. the difference between such curves is in their way of mapping to the one-dimensional space. selecting the appropriate curve for any application requires a brief knowledge of the mapping scheme provided by each space-filling curve. irregularity is proposed as a quantitative measure of the quality of the mapping of the space-filling curve. closed formulas are developed to compute the irregularity for any general dimension d with n points in each dimension for different space-filling curves.a comparative study of different space-filling curves with respect to irregularity is conducted and results are presented and discussed. the applicability of this research is the area of multimedia databases is illustrated with a discussion of the problems that arise.
relevance score normalization for metasearch. given the ranked lists of documents returned by multiple search engines in response to a given query, the problem of metasearch is to combine these lists in a way which optimizes the performance of the combination. this problem can be naturally decomposed into three subproblems: (1) normalizing the relevance scores given by the input systems, (2) estimating relevance scores for unretrieved documents, and (3) combining the newly-acquired scores for each document into one, improved score.research on the problem of metasearch has historically concentrated on algorithms for combining (normalized) scores. in this paper, we show that the techniques used for normalizing relevance scores and estimating the relevance scores of unretrieved documents can have a significant effect on the overall performance of metasearch. we propose two new normalization/estimation techniques and demonstrate empirically that the performance of well known metasearch algorithms can be significantly improved through their use.
citeseer-api: towards seamless resource location and interlinking for digital libraries. we introduce citeseer-api, a public api to citeseer-like services. citeseer-api is soap/wsdl based and allows for easy programmatical access to all the specific functionalities offered by citeseer services, including full text search of documents and citations and citation-based document discovery. in order to enable operability and interlinking with arbitrary software agents and digital library systems, citeseer-api uses digital content signatures to create system-independent handles for the document, citation and group resources of citeseer servers. we discuss specific functionalities of citeseer-api that take advantage of these handlers in order to enable seamless location of citeseer resources. finally we argue that the digital signature scheme used by citeseer-api is well suited for the creation of machine-usable semantic descriptions of digital library services which is the key toward seamless discovery and integration of services such as citeseer-api. citeseer-api is currently showcased on citeseer.ist, the citeseer server of the school of information science and technology at the pennsylvania state university.
condorcet fusion for improved retrieval. we present a new algorithm for improving retrieval results by combining document ranking functions: condorcet-fuse. beginning with one of the two major classes of voting procedures from social choice theory, the condorcet procedure, we apply a graph-theoretic analysis that yields a sorting-based algorithm that is elegant, efficient, and effective. the algorithm performs very well on trec data, often outperforming existing metasearch algorithms whether or not relevance scores and training data is available. condorcet-fuse significantly outperforms borda-fuse, the analogous representative from the other major class of voting algorithms.
harmonic models for polyphonic music retrieval. most work in the ad hoc music retrieval field has focused on the retrieval of monophonic documents using monophonic queries. polyphony adds considerably more complexity. we present a method by which polyphonic music documents may be retrieved by polyphonic music queries. a new harmonic description technique is given, wherein the information from all chords, rather than the most significant chord, is used. this description is then combined in a new and unique way with markov statistical methods to create models of both documents and queries. document models are compared to query models and then ranked by score. though test collections for music are currently scarce, we give the first known recall-precision graphs for polyphonic music retrieval, and results are favorable.
mining temporal classes from time series data. in this investigation, we discuss how to mine temporal class schemes to model a collection of time series data. from the viewpoint of temporal data mining, this problem can be seen as discretizing time series data or aggregating them. also this can be considered as screening (or noise filtering). from the viewpoint of temporal databases, the issue is how we represent the data and how we can obtain intensional aspects as temporal schemes. in other words, we discuss scheme discovery for temporal data. given a collection of temporal objects along with time axis (called log), we examine the data and we introduce a notion of temporal frequent classes to describe them. as the main results of this investigation, we can show that there exists one and only one interval decomposition and the temporal classes related to them. also we give experimental results that prove the feasibility to time series data.
multi-dimensional sequential pattern mining. sequential pattern mining, which finds the set of frequent subsequences in sequence databases, is an important data-mining task and has broad applications. usually, sequence patterns are associated with different circumstances, and such circumstances form a multiple dimensional space. for example, customer purchase sequences are associated with region, time, customer group, and others. it is interesting and useful to mine sequential patterns associated with multi-dimensional information.in this paper, we propose the theme of multi-dimensional sequential pattern mining, which integrates the multidimensional analysis and sequential data mining. we also thoroughly explore efficient methods for multi-dimensional sequential pattern mining. we examine feasible combinations of efficient sequential pattern mining and multi-dimensional analysis methods, as well as develop uniform methods for high-performance mining. extensive experiments show the advantages as well as limitations of these methods. some recommendations on selecting proper method with respect to data set properties are drawn.
efficient evaluation of parameterized pattern queries. many applications rely on sequence databases and use extensively pattern-matching queries to retrieve data of interest. this paper extends the traditional pattern-matching expressions to parameterized patterns, featuring variables. parameterized patterns are more expressive and allow to define concisely regular expressions that would be very complex to describe without variables. they can also be used to express additional constraints on patterns' variables.we show that they can be evaluated without additional cost with respect to traditional techniques (e.g., the knuth-morris-pratt algorithm). we describe an algorithm that enjoys low memory and cpu time requirements, and provide experimental results which illustrate the gain of the optimized solution.
energy management schemes for memory-resident database systems. with the tremendous growth of system memories, memory-resident databases are increasingly becoming important in various domains. newer memories provide a structured way of storing data in multiple chips, with each chip having a bank of memory modules. current memory-resident databases are yet to take full advantage of the banked storage system, which offers a lot of room for performance and energy optimizations. in this paper, we identify the implications of a banked memory environment in supporting memory-resident databases, and propose hardware (memory-directed) and software (query-directed) schemes to reduce the energy consumption of queries executed on these databases. our results show that high-level query-directed schemes (hosted in the query optimizer) better utilize the low-power modes in reducing the energy consumption than the respective hardware schemes (hosted in the memory controller), due to their complete knowledge of query access patterns. we extend this further and propose a query restructuring scheme and a multi-query optimization. queries are restructured and regrouped based on their table access patterns to maximize the likelihood that data accesses are clustered. this helps increase the inter-access idle times of memory modules, which in turn enables a more effective control of their energy behavior. this heuristic is eventually integrated with our hardware optimizations to achieve maximum savings. our experimental results show that the memory energy reduces by 90% if query restructuring method is applied along with basic energy optimizations over the unoptimized version. the system-wide performance impact of each scheme is also studied simultaneously.
symbolic photograph content-based retrieval. photograph retrieval systems face the difficulty to deal with the different ways to apprehend the content of images. we consider and demonstrate here the use of multiple index representations of photographs to achieve effective retrieval. the use of multiple indexes allows integration of the complementary strengths of different indexing and retrieval models. the proposed representation supports multiple labels for regions and attributes, and handles inferences and relationships. we define links between indexing levels and the related query modes. the experiment conducted on 2400 home photographs shows the behavior of the multiple indexing levels during retrieval.
scalable summary based retrieval in p2p networks. much of the present p2p-ir literature is focused on distributed indexing structures. within this paper, we present an approach based on the replication of peer data summaries via rumor spreading and multicast in a structured overlay.we will describe rumorama, a p2p framework for similar-ity queries inspired by gloss and cori and their p2p-adaptation, planetp. rumorama achieves a hierarchization of planetp-like summary-based p2p-ir networks. in a rumorama network, each peer views the network as a small planetp network with connections to peers that see other small planetp networks. one important aspect is that each peer can choose the size of the planetp network it wants to see according to its local processing power and bandwidth. even in this adaptive environment, rumorama manages to process a query such that the summary of each peer is considered exactly once in a network without churn. however, the actual number of peers to be contacted for a query is a small fraction of the total number of peers in the network.within this article, we present the rumorama base protocol, as well as experiments demonstrating the scalability and viability of the approach under churn.