OpenRaiser/Intern-Atlas huggingface dataset:
HyperGenFL: Hypernetwork-generated model aggregation in federated learning
Authors: Jerry Chen, Qikai Lu, Ruiqing Tian, Di Niu, Baochun Li
Venue: Conference CIKM 2025
Year: 2025

Federated learning is a decentralized framework that enables client participation in collaborative learning without centralized data collection. However, the framework is susceptible to suboptimal model convergence induced by heterogeneity among the client datasets. These discrepancies, including label imbalance, dissimilarity in data distributions, and uneven data volumes between clients, may cause disagreements among local client updates, affecting the ability of the global model to converge effectively during aggregation. We suggest that one potential solution to this problem lies in weighting the model aggregation by client importance and client-to-client relationships. Based on this idea, we propose HyperGenFL (HG-FL), a hypernetwork that generates aggregation weights from learnable client embeddings without requiring any training or benchmarking data. HG-FL utilizes the attention mechanism to capture inter-client relationships based on learnable client-specific embeddings in order to generate model aggregation weights dynamically during federated learning. By guiding the aggregation process with these learnable relationships between local models, HG-FL reduces update conflicts and improves global model performance. We assess HG-FL under various data-heterogeneous environments based on different benchmark datasets including Fashion-MNIST, CIFAR10, CIFAR100 and Tiny-ImageNet. Experimental results demonstrate that HG-FL can achieve superior performance over a range of existing baseline methods under challenging cases with various heterogeneous environments, large models and a large number of clients.
S-tree: A dynamic Balanced signature Index for Office Retrieval
Authors: Uwe Deppisch
Venue: Conference SIGIR 1986
Year: 1986

The signature approach is an access method for partial-match retrieval which meets many requirements of an office environment. Signatures are hash coded binary words derived from objects stored in the data base. They serve as a filter for retrieval in order to discard a large number of nonqualifying objects. In an indexed signature method the signatures of objects stored on a single page are used to form a signature for that page. In this paper we describe a new technique of indexed signatures which combines the dynamic balancing of B-trees with the signature approach. The main problem of appropriate splitting is solved in a heuristic way. Operations are described and a simple performance analysis is given. The analysis and some experimental results indicate a considerable performance gain. Moreover, the new S-tree approach supports a clustering on a signature basis. Further remarks on adaptability complete this work.
A Study on Thresholding Strategies for Text Categorization
Authors: Yiming Yang
Venue: Conference SIGIR 2001
Year: 2001

Thresholding strategies in automated text categorization are an underexplored area of research. This paper presents an examination of the effect of thresholding strategies on the performance of a classifier under various conditions. Using k-Nearest Neighbor (kNN) as the classifier and five evaluation benchmark collections as the testbets, three common thresholding methods were investigated, including rank-based thresholding (RCut), proportion-based assignments (PCut) and score-based local optimization (SCut); in addition, new variants of these methods are proposed to overcome significant problems in the existing approaches. Experimental results show that the choice of thresholding strategy can significantly influence the performance of kNN, and that the ``optimal'' strategy may vary by application. SCut is potentially better for fine-tuning but risks overfitting. PCut copes better with rare categories and exhibits a smoother trade-off in recall versus precision, but is not suitable for online decision making. RCut is most natural for online response but is too coarse-grained for global or local optimization. RTCut, a new method combining the strength of category ranking and scoring, outperforms both PCut and RCut significantly.
Visualizing Multi-Dimensional Clusters, Trends, and Outliers Using Star Coordinates
Authors: Eser Kandogan
Venue: Conference KDD 2001
Year: 2001

Interactive visualizations are effective tools in mining scientific, engineering, and business data to support decision-making activities. Star Coordinates is proposed as a new multi-dimensional visualization technique, which supports various interactions to stimulate visual thinking in early stages of knowledge discovery process. In Star Coordinates, coordinate axes are arranged on a two-dimensional surface, where each axis shares the same origin point. Each multi-dimensional data element is represented by a point, where each attribute of the data contributes to its location through uniform encoding. Interaction features of Star Coordinates provide users the ability to apply various transformations dynamically, integrate and separate dimensions, analyze correlations of multiple dimensions, view clusters, trends, and outliers in the distribution of data, and query points based on data ranges. Our experience with Star Coordinates shows that it is particularly useful for the discovery of hierarchical clusters, and analysis of multiple factors providing insight in various real datasets including telecommunications churn.
Discovering Significant Rules
Authors: Geoffrey I. Webb
Venue: Conference KDD 2006
Year: 2006

In many applications, association rules will only be interesting if they represent non-trivial correlations between all constituent items. Numerous techniques have been developed that seek to avoid false discoveries. However, while all provide useful solutions to aspects of this problem, none provides a generic solution that is both flexible enough to accommodate varying definitions of true and false discoveries and powerful enough to provide strict control over the risk of false discoveries. This paper presents generic techniques that allow definitions of true and false discoveries to be specified in terms of arbitrary statistical hypothesis tests and which provide strict control over the experiment wise risk of false discoveries.
Frequent Pattern Mining with Uncertain Data
Authors: Charu C. Aggarwal, Yan Li, Jianyong Wang, Jing Wang
Venue: Conference KDD 2009
Year: 2009

This paper studies the problem of frequent pattern mining with uncertain data. We will show how broad classes of algorithms can be extended to the uncertain data setting. In particular, we will study candidate generate-and-test algorithms, hyper-structure algorithms and pattern growth based algorithms. One of our insightful observations is that the experimental behavior of different classes of algorithms is very different in the uncertain case as compared to the deterministic case. In particular, the hyper-structure and the candidate generate-and-test algorithms perform much better than tree-based algorithms. This counter-intuitive behavior is an important observation from the perspective of algorithm design of the uncertain variation of the problem. We will test the approach on a number of real and synthetic data sets, and show the effectiveness of two of our approaches over competitive techniques.
Mining Frequent Neighboring Class Sets in Spatial Databases
Authors: Yasuhiko Morimoto
Venue: Conference KDD 2001
Year: 2001

We consider the problem of finding neighboring class sets. Objects of each instance of a neighboring class set are grouped using their Euclidean distances from each other. Recently, location-based services are growing along with mobile computing infrastructure such as cellular phones and PDAs. Therefore, we expect to see the development of spatial databases that contains very large number of access records including location information. The most typical type would be a database of point objects. Records of the objects may consist of "requested service name," "number of packet transmitted" in addition to x and y coordinate values indicating where the request came from. The algorithm presented here efficiently finds sets of "service names" that were frequently close to each other in the spatial database. For example, it may find a frequent neighboring class set, where "ticket" and "timetable" are frequently requested close to each other. By recognizing this, location-based service providers can promote a "ticket" service for customers who access the "timetable."
Improving novelty detection for general topics using sentence level information patterns
Authors: Xiaoyan Li, W. Bruce Croft
Venue: Conference CIKM 2006
Year: 2006

The detection of new information in a document stream is an important component of many potential applications. In this work, a new novelty detection approach based on the identification of sentence level information patterns is proposed. First, the information-pattern concept for novelty detection is presented with the emphasis on new information patterns for general topics (queries) that cannot be simply turned into specific questions whose answers are specific named entities (NEs). Then we elaborate a thorough analysis of sentence level information patterns on data from the TREC novelty tracks, including sentence lengths, named entities, sentence level opinion patterns. This analysis provides guidelines in applying those patterns in novelty detection particularly for the general topics. Finally, a unified pattern-based approach is presented to novelty detection for both general and specific topics. The new method for dealing with general topics will be the focus. Experimental results show that the proposed approach significantly improves the performance of novelty detection for general topics as well as the overall performance for all topics from the 2002-2004 TREC novelty tracks.
Delta-SimRank computing on MapReduce
Authors: Liangliang Cao, Brian Cho, Hyun Duk Kim, Zhen Li, Min-Hsuan Tsai, Indranil Gupta
Venue: Conference KDD 2012
Year: 2012

Based on the intuition that "two objects are similar if they are related to similar objects", SimRank (proposed by Jeh and Widom in 2002) has become a famous measure to compare the similarity between two nodes using network structure. Although SimRank is applicable to a wide range of areas such as social networks, citation networks, link prediction, etc., it suffers from heavy computational complexity and space requirements. Most existing efforts to accelerate SimRank computation work only for static graphs and on single machines. This paper considers the problem of computing SimRank efficiently in a distributed system while handling dynamic networks which grow with time. We first consider an abstract model called Harmonic Field on Node-pair Graph. We use this model to derive SimRank and the proposed Delta-SimRank, which is demonstrated to fit the nature of distributed computing and can be efficiently implemented using Google's MapReduce paradigm. Delta-SimRank can effectively reduce the computational cost and can also benefit the applications with non-static network structures. Our experimental results on four real world networks show that Delta-SimRank is much more efficient than the distributed SimRank algorithm, and leads to up to 30 times speed-up in the best case1.
Markov logic: A unifying language for knowledge and information management
Authors: Pedro M. Domingos
Venue: Conference CIKM 2008
Year: 2008

Modern information and knowledge management is characterized by high degrees of complexity and uncertainty. Complexity is well handled by first-order logic, and uncertainty by probabilistic graphical models. What has been sorely missing is a seamless combination of the two. Markov logic provides this by attaching weights to logical formulas and treating them as templates for features of Markov random fields. This talks surveys Markov logic representation, inference, learning and applications. Inference algorithms combine ideas from satisfiability testing, resolution, Markov chain Monte Carlo and belief propagation. Learning algorithms involve statistical weight learning and inductive logic programming. Markov logic has been successfully applied to a wide range of information and knowledge management problems, including information extraction, entity resolution, ontology learning, link prediction, heterogeneous knowledge bases, and others. It is the basis of the open-source Alchemy system (http://alchemy.cs.washington.edu).
Predictive parallelization: Taming tail latencies in Web search
Authors: Myeongjae Jeon, Saehoon Kim, Seung-won Hwang, Yuxiong He, Sameh Elnikety, Alan L. Cox, Scott Rixner
Venue: Conference SIGIR 2014
Year: 2014

Web search engines are optimized to reduce the high-percentile response time to consistently provide fast responses to almost all user queries. This is a challenging task because the query workload exhibits large variability, consisting of many short-running queries and a few long-running queries that significantly impact the high-percentile response time. With modern multicore servers, parallelizing the processing of an individual query is a promising solution to reduce query execution time, but it gives limited benefits compared to sequential execution since most queries see little or no speedup when parallelized. The root of this problem is that short-running queries, which dominate the workload, do not benefit from parallelization. They incur a large parallelization overhead, taking scarce resources from long-running queries. On the other hand, parallelization substantially reduces the execution time of long-running queries with low overhead and high parallelization efficiency. Motivated by these observations, we propose a predictive parallelization framework with two parts: (1) predicting long-running queries, and (2) selectively parallelizing them. For the first part, prediction should be accurate and efficient. For accuracy, we study a comprehensive feature set covering both term features (reflecting dynamic pruning efficiency) and query features (reflecting query complexity). For efficiency, to keep overhead low, we avoid expensive features that have excessive requirements such as large memory footprints. For the second part, we use the predicted query execution time to parallelize long-running queries and process short-running queries sequentially. We implement and evaluate the predictive parallelization framework in Microsoft Bing search. Our measurements show that under moderate to heavy load, the predictive strategy reduces the 99th-percentile response time by 50% (from 200 ms to 100 ms) compared with prior approaches that parallelize all queries.
STED: Semi-supervised targeted-interest event detectionin in Twitter
Authors: Ting Hua, Feng Chen, Liang Zhao, Chang-Tien Lu, Naren Ramakrishnan
Venue: Conference KDD 2013
Year: 2013

Social microblogs such as Twitter and Weibo are experiencing an explosive growth with billions of global users sharing their daily observations and thoughts. Beyond public interests (e.g., sports, music), microblogs can provide highly detailed information for those interested in public health, homeland security, and financial analysis. However, the language used in Twitter is heavily informal, ungrammatical, and dynamic. Existing data mining algorithms require extensive manually labeling to build and maintain a supervised system. This paper presents STED, a semi-supervised system that helps users to automatically detect and interactively visualize events of a targeted type from twitter, such as crimes, civil unrests, and disease outbreaks. Our model first applies transfer learning and label propagation to automatically generate labeled data, then learns a customized text classifier based on mini-clustering, and finally applies fast spatial scan statistics to estimate the locations of events. We demonstrate STED's usage and benefits using twitter data collected from Latin America countries, and show how our system helps to detect and track example events such as civil unrests and crimes.
An information theoretic framework for data mining
Authors: Tijl De Bie
Venue: Conference KDD 2011
Year: 2011

We formalize the data mining process as a process of information exchange, defined by the following key components. The data miner's state of mind is modeled as a probability distribution, called the background distribution, which represents the uncertainty and misconceptions the data miner has about the data. This model initially incorporates any prior (possibly incorrect) beliefs a data miner has about the data. During the data mining process, properties of the data (to which we refer as patterns) are revealed to the data miner, either in batch, one by one, or even interactively. This acquisition of information in the data mining process is formalized by updates to the background distribution to account for the presence of the found patterns.
Ranking interesting subgroups
Authors: Stefan Rüping 0001
Venue: Conference ICML 2009
Year: 2009

Subgroup discovery is the task of identifying the top k patterns in a database with most significant deviation in the distribution of a target attribute Y. Subgroup discovery is a popular approach for identifying interesting patterns in data, because it combines statistical significance with an understandable representation of patterns as a logical formula. However, it is often a problem that some subgroups, even if they are statistically highly significant, are not interesting to the user. We present an approach based on the work on ranking Support Vector Machines that ranks subgroups with respect to the user's concept of interestingness, and finds more interesting subgroups. This approach can significantly increase the quality of the subgroups.
Detecting Time Series Motifs under Uniform Scaling
Authors: Dragomir Yankov, Eamonn J. Keogh, Jose Medina, Bill Yuan-chi Chiu, Victor B. Zordan
Venue: Conference KDD 2007
Year: 2007

Time series motifs are approximately repeated patterns foundwithin the data. Such motifs have utility for many data mining algorithms, including rule-discovery,novelty-detection, summarization and clustering. Since the formalization of the problem and the introduction of efficient linear time algorithms, motif discovery has been successfully applied tomany domains, including medicine, motion capture, robotics and meteorology.In this work we show that most previous applications of time series motifs have been severely limited by the definition's brittleness to even slight changes of uniform scaling, the speed at which the patterns develop. We introduce a new algorithm that allows discovery of time series motifs with invariance to uniform scaling, and show that it produces objectively superior results in several important domains. Apart from being more general than all other motifdiscovery algorithms, a further contribution of our work isthat it is simpler than previous approaches, in particular we have drastically reduced the number of parameters that need to be specified.
City-scale localization with telco big data
Authors: Fangzhou Zhu, Chen Luo, Mingxuan Yuan, Yijian Zhu, Zhengqing Zhang, Tao Gu, Ke Deng, Weixiong Rao, Jia Zeng
Venue: Conference CIKM 2016
Year: 2016

It is still challenging in telecommunication (telco) industry to accurately locate mobile devices (MDs) at city-scale using the measurement report (MR) data, which measure parameters of radio signal strengths when MDs connect with base stations (BSs) in telco networks for making/receiving calls or mobile broadband (MBB) services. In this paper, we find that the widely-used location based services (LBSs) have accumulated lots of over-the-top (OTT) global positioning system (GPS) data in telco networks, which can be automatically used as training labels for learning accurate MR-based positioning systems. Benefiting from these telco big data, we deploy a context-aware coarse-to-fine regression (CCR) model in Spark/Hadoop-based telco big data platform for city-scale localization of MDs with two novel contributions. First, we design map-matching and interpolation algorithms to encode contextual information of road networks. Second, we build a two-layer regression model to capture coarse-to-fine contextual features in a short time window for improved localization performance. In our experiments, we collect 108 GPS-associated MR records in the centroid of Shanghai city with 12 x 11 square kilometers for 30 days, and measure four important properties of real-world MR data related to localization errors: stability, sensitivity, uncertainty and missing values. The proposed CCR works well under different properties of MR data and achieves a mean error of 110m and a median error of $80m$, outperforming the state-of-art range-based and fingerprinting localization methods.
Broad learning based multi-source collaborative recommendation
Authors: Junxing Zhu, Jiawei Zhang, Lifang He, Quanyuan Wu, Bin Zhou, Chenwei Zhang, Philip S. Yu
Venue: Conference CIKM 2017
Year: 2017

Anchor links connect information entities, such as entities of movies or products, across networks from different sources, and thus information in these networks can be transferred directly via anchor links. Therefore, anchor links have great value to many cross-network applications, such as cross-network social link prediction and cross-network recommendation. In this paper, we focus on studying the recommendation problem that can provide ratings of items or services. To address the problem, we propose a Cross-network Collaborative Matrix Factorization (CCMF) recommendation framework based on broad learning setting, which can effectively integrate multi-source information and alleviate the sparse information problem in each individual network. Based on item anchor links CCMF can fuse item similarity information and item latent information across networks from different sources. And different from most of the traditional works, CCMF can make multi-source recommendation tasks collaborate together via the information transfer based on the broad learning setting. During the transfer process, a novel cross-network similarity transfer method is applied to keep the consistency of item similarities between two different networks, and a domain adaptation matrix is used to overcome the domain difference problem. We conduct experiments to compare the proposed CCMF method with both classic and state-of-the-art recommendation techniques. The experimental results illustrate that CCMF outperforms other methods in different experimental circumstances, and has great advantages on dealing with different data sparse problems.