OpenRaiser/Intern-Atlas huggingface dataset:
Into the Battlefield: Quantifying and Modeling Intra-community Conflicts in Online Discussion
Authors: Subhabrata Dutta, Dipankar Das, Gunkirat Kaur, Shreyans Mongia, Arpan Mukherjee, Tanmoy Chakraborty
Venue: Conference CIKM 2019
Year: 2019

Over the last decade, online forums have become primary news sources for readers around the globe, and social media platforms are the space where these news forums find most of their audience and engagement. Our particular focus in this paper is to study conflict dynamics over online news articles in Reddit, one of the most popular online discussion platforms. We choose to study how conflicts develop around news inside a discussion community, the \em r/news subreddit. Mining the characteristics of these engagements often provide useful insights into the behavioral dynamics of large-scale human interactions. Such insights are useful for many reasons -- for news houses to improvise their publishing strategies and potential audience, for data analytics to get a better introspection over media engagement as well as for social media platforms to avoid unnecessary and perilous conflicts. In this work, we present a novel quantification of conflict in online discussion. Unlike previous studies on conflict dynamics, which model conflict as a binary phenomenon, our measure is continuous-valued, which we validate with manually annotated ratings. We address a two-way prediction task. Firstly, we predict the probable degree of conflict a news article will face from its audience. We employ multiple machine learning frameworks for this task using various features extracted from news articles.Secondly, given a pair of users and their interaction history, we predict if their future engagement will result in a conflict. We fuse textual and network-based features together using a support vector machine which achieves an AUC of 0.89. Moreover, we implement a graph convolutional model which exploits engagement histories of users to predict whether a pair of users who never met each other before will have a conflicting interaction, with an AUC of 0.69. We perform our studies on a massive discussion dataset crawled from the Reddit news community, containing over $41k$ news articles and $5.5$ million comments. Apart from the prediction tasks, our studies offer interesting insights on the conflict dynamics -- how users form clusters based on conflicting engagements, how different is the temporal nature of conflict over different online news forums, how is contribution of different language based features to induce conflict, etc. In short, our study paves the way towards new methods of exploration and modeling of conflict dynamics inside online discussion communities.
Pass: Personalized advertiser-aware sponsored search
Authors: Zhoujin Tian, Chaozhuo Li, Zhiqiang Zuo 0004, Zengxuan Wen, Lichao Sun 0001, Xinyue Hu 0003, Wen Zhang, Haizhen Huang, Senzhang Wang, Weiwei Deng, Xing Xie 0001, Qi Zhang 0066
Venue: Conference KDD 2023
Year: 2023

The nucleus of online sponsored search systems lies in measuring the relevance between the search intents of users and the advertising purposes of advertisers. Existing conventional doublet-based (query-keyword) relevance models solely rely on short queries and keywords to uncover such intents, which ignore the diverse and personalized preferences of participants (i.e., users and advertisers), resulting in undesirable advertising performance. In this paper, we investigate the novel problem of Personalized A dvertiser-aware Sponsored Search (PASS). Our motivation lies in incorporating the portraits of users and advertisers into relevance models to facilitate the modeling of intrinsic search intents and advertising purposes, leading to a quadruple-based (i.e., user-query-keyword-advertiser) task. Various types of historical behaviors are explored in the format of hypergraphs to provide abundant signals on identifying the preferences of participants. A novel heterogeneous textual hypergraph transformer is further proposed to deeply fuse the textual semantics and the high-order hypergraph topology. Our proposal is extensively evaluated over real industry datasets, and experimental results demonstrate its superiority.
Top-k retrieval using conditional preference networks
Authors: Hongbing Wang, Xuan Zhou, Wujin Chen, Peisheng Ma
Venue: Conference CIKM 2012
Year: 2012

This paper considers top-k retrieval using Conditional Preference Network (CP-Net). As a model for expressing user preferences on multiple mutually correlated attributes, CP-Net is of great interest for decision support systems. However, little work has addressed how to conduct efficient data retrieval using CP-Nets. This paper presents an approach to efficiently retrieve the most preferred data items based on a user's CP-Net. The proposed approach consists of a top-k algorithm and an indexing scheme. We conducted extensive experiments to compare our approach against a baseline top-k method - sequential scan. The results show that our approach outperform sequential scan in several circumstances.
Portfolio optimization for influence spread
Authors: Naoto Ohsaka, Yuichi Yoshida
Venue: Conference WWW 2017
Year: 2017

Motivated by viral marketing, stochastic diffusion processes that model influence spread on a network have been studied intensively. The primary interest in such models has been to find a seed set of a fixed size that maximizes the expected size of the cascade from it. Practically, however, it is not desirable to have the risk of ending with a small cascade, even if the expected size of the cascade is large. To address this issue, we adopt conditional value at risk (CVaR) as a risk measure, and propose an algorithm that computes a portfolio over seed sets with a provable guarantee on its CVaR. Using real-world social networks, we demonstrate that the portfolio computed by our algorithm has a significantly better CVaR than seed sets computed by other baseline methods.
Another Look at Information Retrieval as Statistical Translation
Authors: Yuqi Liu, Chengcheng Hu, Jimmy Lin
Venue: Conference SIGIR 2022
Year: 2022

Over two decades ago, Berger and Lafferty proposed "information retrieval as statistical translation" (IRST), a simple and elegant method for ad hoc retrieval based on the noisy channel model. At the time, they lacked the large-scale human-annotated datasets necessary to properly train their models. In this paper, we ask the simple question: What if Berger and Lafferty had access to datasets such as the MS MARCO passage ranking dataset that we take for granted today? The answer to this question tells us how much of recent improvements in ranking can be solely attributed to having more data available, as opposed to improvements in models (e.g., pretrained transformers) and optimization techniques (e.g., contrastive loss). In fact, Boytsov and Kolter recently began to answer this question with a replication of Berger and Lafferty's model, and this work can be viewed as another independent replication effort, with generalizations to additional conditions not previously explored, including replacing the sum of translation probabilities with ColBERT's MaxSim operator. We confirm that while neural models (particularly pretrained transformers) have indeed led to great advances in retrieval effectiveness, the IRST model proposed decades ago is quite effective if provided sufficient training data.
Users and Contemporary SERPs: A (Re-)Investigation
Authors: Nirmal Roy, David Maxwell, Claudia Hauff
Venue: Conference SIGIR 2022
Year: 2022

TheSearch Engine Results Page (SERP) has evolved significantly over the last two decades, moving away from the simple ten blue links paradigm to considerably more complex presentations that contain results from multiple verticals and granularities of textual information. Prior works have investigated how user interactions on the SERP are influenced by the presence or absence of heterogeneous content (e.g., images, videos, or news content), the layout of the SERP (\emphlist vs. grid layout), and task complexity. In this paper, we reproduce the user studies conducted in prior works---specifically those of~\citetarguello2012task and~\citetsiu2014first ---to explore to what extent the findings from research conducted five to ten years ago still hold today as the average web user has become accustomed to SERPs with ever-increasing presentational complexity. To this end, we designed and ran a user study with four different SERP interfaces:(i) ~\empha heterogeneous grid ;(ii) ~\empha heterogeneous list ;(iii) ~\empha simple grid ; and(iv) ~\empha simple list. We collected the interactions of $41$ study participants over $12$ search tasks for our analyses. We observed that SERP types and task complexity affect user interactions with search results. We also find evidence to support most (6 out of 8) observations from~\citearguello2012task,siu2014first indicating that user interactions with different interfaces and to solve tasks of different complexity have remained mostly similar over time.
Towards model-agnostic post-hoc adjustment for balancing ranking fairness and algorithm utility
Authors: Sen Cui, Weishen Pan, Changshui Zhang, Fei Wang
Venue: Conference KDD 2021
Year: 2021

Bipartite ranking, which aims to learn a scoring function that ranks positive individuals higher than negative ones from labeled data, is widely adopted in various applications where sample prioritization is needed. Recently, there have been rising concerns on whether the learned scoring function can cause systematic disparity across different protected groups defined by sensitive attributes. While there could be trade-off between fairness and performance, in this paper we propose a model agnostic post-processing framework for balancing them in the bipartite ranking scenario. Specifically, we maximize a weighted sum of the utility and fairness by directly adjusting the relative ordering of samples across groups. By formulating this problem as the identification of an optimal warping path across different protected groups, we propose a non-parametric method to search for such an optimal path through a dynamic programming process. Our method is compatible with various classification models and applicable to a variety of ranking fairness metrics. Comprehensive experiments on a suite of benchmark data sets and two real-world patient electronic health record repositories show that our method can achieve a great balance between the algorithm utility and ranking fairness. Furthermore, we experimentally verify the robustness of our method when faced with the fewer training samples and the difference between training and testing ranking score distributions.
GPFedRec: Graph-Guided Personalization for Federated Recommendation
Authors: Chunxu Zhang, Guodong Long, Tianyi Zhou 0001, Zijian Zhang 0009, Peng Yan 0002, Bo Yang 0002
Venue: Conference KDD 2024
Year: 2024

The federated recommendation system is an emerging AI service architecture that provides recommendation services in a privacy-preserving manner. Using user-relation graphs to enhance federated recommendations is a promising topic. However, it is still an open challenge to construct the user-relation graph while preserving data locality-based privacy protection in federated settings. Inspired by a simple motivation, similar users share a similar vision (embeddings) to the same item set, this paper proposes a novel Graph-guided Personalization for Federated Recommendation (GPFedRec). The proposed method constructs a user-relation graph from user-specific personalized item embeddings at the server without accessing the users' interaction records. The personalized item embedding is locally fine-tuned on each device, and then a user-relation graph will be constructed by measuring the similarity among client-specific item embeddings. Without accessing users' historical interactions, we embody the data locality-based privacy protection of vanilla federated learning. Furthermore, a graph-guided aggregation mechanism is designed to leverage the user-relation graph and federated optimization framework simultaneously. Extensive experiments on five benchmark datasets demonstrate GPFedRec's superior performance. The in-depth study validates that GPFedRec can generally improve existing federated recommendation methods as a plugin while keeping user privacy safe. Code is available https://github.com/Zhangcx19/GPFedRec
Healai: A healthcare llm for effective medical documentation
Authors: Sagar Goyal, Eti Rastogi, Sree Prasanna Rajagopal, Dong Yuan, Fen Zhao, Jai Chintagunta, Gautam Naik, Jeff Ward
Venue: Conference WSDM 2024
Year: 2024

Since the advent of LLM's like GPT4 everyone in various industries has been trying to harness their power. Healthcare is an industry where this is a specifically challenging problem due to the high accuracy requirements. Prompt Engineering is a common technique used to design instructions for model responses, however, its challenges lie in the fact that the generic models may not be trained to accurately execute these specific tasks. We will present our journey of developing a cost-effective medical LLM, surpassing GPT4 in medical note-writing tasks. We'll touch upon our trials with medical prompt engineering, GPT4's limitations, and training an optimized LLM for specific medical tasks. We'll showcase multiple comparisons on model sizes, training data, and pipeline designs that enabled us to outperform GPT4 with smaller models, maintaining precision, reducing biases, preventing hallucinations, and enhancing note-writing style.
Packet Representation Learning for Traffic Classification
Authors: Xuying Meng, Yequan Wang, Runxin Ma, Haitong Luo, Xiang Li, Yujun Zhang 0001
Venue: Conference KDD 2022
Year: 2022

With the surging development of information technology, to provide a high quality of network services, there are increasing demands and challenges for network analysis. As all data on the Internet are encapsulated and transferred by network packets, packets are widely used for various network traffic analysis tasks, from application identification to intrusion detection. Considering the choice of features and how to represent them can greatly affect the performance of downstream tasks, it is critical to learn high-quality packet representations. In addition, existing packet-level works ignore packet representations but focus on trying to get good performance with independent analysis of different classification tasks. In the real world, although a packet may have different class labels for different tasks, the packet representation learned from one task can also help understand its complex packet patterns in other tasks, while existing works omit to leverage them.
Glad: group anomaly detection in social media analysis
Authors: Qi Rose Yu, Xinran He, Yan Liu
Venue: Conference KDD 2014
Year: 2014

Traditional anomaly detection on social media mostly focuses on individual point anomalies while anomalous phenomena usually occur in groups. Therefore it is valuable to study the collective behavior of individuals and detect group anomalies. Existing group anomaly detection approaches rely on the assumption that the groups are known, which can hardly be true in real world social media applications. In this paper, we take a generative approach by proposing a hierarchical Bayes model: Group Latent Anomaly Detection (GLAD) model. GLAD takes both pair-wise and point-wise data as input, automatically infers the groups and detects group anomalies simultaneously. To account for the dynamic properties of the social media data, we further generalize GLAD to its dynamic extension d-GLAD. We conduct extensive experiments to evaluate our models on both synthetic and real world datasets. The empirical results demonstrate that our approach is effective and robust in discovering latent groups and detecting group anomalies.
Towards validating long-term user feedbacks in interactive recommendation systems
Authors: Hojoon Lee, Dongyoon Hwang, Kyushik Min, Jaegul Choo
Venue: Conference SIGIR 2022
Year: 2022

Interactive Recommender Systems (IRSs) have attracted a lot of attention, due to their ability to model interactive processes between users and recommender systems. Numerous approaches have adopted Reinforcement Learning (RL) algorithms, as these can directly maximize users' cumulative rewards. In IRS, researchers commonly utilize publicly available review datasets to compare and evaluate algorithms. However, user feedback provided in public datasets merely includes instant responses (e.g., a rating), with no inclusion of delayed responses (e.g., the dwell time and the lifetime value). Thus, the question remains whether these review datasets are an appropriate choice to evaluate the long-term effects in IRS. In this work, we revisited experiments on IRS with review datasets and compared RL-based models with a simple reward model that greedily recommends the item with the highest one-step reward. Following extensive analysis, we can reveal three main findings: First, a simple greedy reward model consistently outperforms RL-based models in maximizing cumulative rewards. Second, applying higher weighting to long-term rewards leads to degradation of recommendation performance. Third, user feedbacks have mere long-term effects in the benchmark datasets. Based on our findings, we conclude that a dataset has to be carefully verified and that a simple greedy baseline should be included for a proper evaluation of RL-based IRS approaches. Our code and dataset are available at https://github.com/dojeon-ai/irs_validation.
Guessing morphology from terms and corpora
Authors: Christian Jacquemin
Venue: Conference SIGIR 1997
Year: 1997

This study proposes an algorithm for automatically acquiring morphological Iinks between words. This algorithm relies on the concurrent use of a corpus and a list of multi-word terms, and does not require any prior linguistic knowledge. The four steps of the algorithm are (1) single-word truncation, (2) conflation of multi-word terms, (3) classification and filtering, and (4) clustering of contiation clasea. At each step a precise evaluation is performed in order to chose the optimal parameters. The final results indicate a clustering of 45% of the classes with a prectilon of 87Y0. The derivational knowledge acquired through this method can be used for conceiving a domain-oriented stemmer for scientific and technical corpora.
Incorporating quality metrics in centralized/distributed information retrieval on the World Wide Web
Authors: Xiaolan Zhu, Susan Gauch
Venue: Conference SIGIR 2000
Year: 2000

Most information retrieval systems on the Internet rely primarily on similarity ranking algorithms based solely on term frequency statistics. Information quality is usually ignored. This leads to the problem that documents are retrieved without regard to their quality. We present an approach that combines similarity-based similarity ranking with quality ranking in centralized and distributed search environments. Six quality metrics, including the currency, availability, information-to-noise ratio, authority, popularity, and cohesiveness, were investigated. Search effectiveness was significantly improved when the currency, availability, information-to-noise ratio and page cohesiveness metrics were incorporated in centralized search. The improvement seen when the availability, information-to- noise ratio, popularity, and cohesiveness metrics were incorporated in site selection was also significant. Finally, incorporating the popularity metric in information fusion resulted in a significant improvement. In summary, the results show that incorporating quality metrics can generally improve search effectiveness in both centralized and distributed search environments.
Learning domain ontologies for web service descriptions: an experiment in bioinformatics
Authors: Marta Sabou, Chris Wroe, Carole A. Goble, Gilad Mishne
Venue: Conference WWW 2005
Year: 2005

The reasoning tasks that can be performed with semantic web service descriptions depend on the quality of the domain ontologies used to create these descriptions. However, building such domain ontologies is a time consuming and difficult task.We describe an automatic extraction method that learns domain ontologies for web service descriptions from textual documentations attached to web services. We conducted our experiments in the field of bioinformatics by learning an ontology from the documentation of the web services used in myGrid, a project that supports biology experiments on the Grid. Based on the evaluation of the extracted ontology in the context of the project, we conclude that the proposed extraction method is a helpful tool to support the process of building domain ontologies for web service descriptions.
On GMAP: and Other Transformations
Authors: Stephen Robertson
Venue: Conference CIKM 2006
Year: 2006

As an alternative to the usual Mean Average Precision, some use is currently being made of the Geometric Mean Average Precision (GMAP) as a measure of average search effectiveness across topics. GMAP is specifically used to emphasise the lower end of the average precision scale, in order to shed light on poor performance of search engines. This paper discusses the status of this measure and how it should be understood.
A Study of Using an Out-of-Box Commercial MT System for Query Translation in CLIR
Authors: Dan Wu, Daqing He, Heng Ji, Ralph Grishman
Venue: Conference CIKM 2008
Year: 2008

Recent availability of commercial online machine translation (MT) systems makes it possible for layman Web users to utilize the MT capability for cross-language information retrieval (CLIR). To study the effectiveness of using MT for query translation, we conducted a set of experiments using Google Translate, an online MT system provided by Google, for translating queries in CLIR. The experiments show that MT is an excellent tool for the query translation task, and with the help of relevance feedback, it can achieve significant improvement over the monolingual baseline. The MT based query translation not only works for long queries, but is also effective for the short Web queries.
Snare: a link analytic system for graph labeling and risk detection
Authors: Mary McGlohon, Stephen Bay, Markus G. Anderle, David M. Steier, Christos Faloutsos
Venue: Conference KDD 2009
Year: 2009

Classifying nodes in networks is a task with a wide range of applications. It can be particularly useful in anomaly and fraud detection. Many resources are invested in the task of fraud detection due to the high cost of fraud, and being able to automatically detect potential fraud quickly and precisely allows human investigators to work more efficiently. Many data analytic schemes have been put into use; however, schemes that bolster link analysis prove promising. This work builds upon the belief propagation algorithm for use in detecting collusion and other fraud schemes. We propose an algorithm called SNARE (Social Network Analysis for Risk Evaluation). By allowing one to use domain knowledge as well as link knowledge, the method was very successful for pinpointing misstated accounts in our sample of general ledger data, with a significant improvement over the default heuristic in true positive rates, and a lift factor of up to 6.5 (more than twice that of the default heuristic). We also apply SNARE to the task of graph labeling in general on publicly-available datasets. We show that with only some information about the nodes themselves in a network, we get surprisingly high accuracy of labels. Not only is SNARE applicable in a wide variety of domains, but it is also robust to the choice of parameters and highly scalable-linearly with the number of edges in a graph.