OpenRaiser/Intern-Atlas huggingface dataset:
Investigating Natural and Artificial Dynamics in Graph Data Mining and Machine Learning
Authors: Dongqi Fu
Venue: CIKM
Year: 2023

The complexity of relationships between entities is increasing in the era of big data, leading to a growing interest in graph (network) data, owing to its ability to encode intricate relational information. Graph data mining and machine learning methods extract informative node and graph representations to support broad applications, which have been proven effective for various high-impact tasks in the fields such as computer vision, natural language processing, and recommendation systems. Despite their effectiveness, graph data mining and machine learning methods face practical challenges in real-world scenarios. First, the input graphs may change over time, making it necessary to integrate time-evolving information to enhance representation capabilities. Second, the input graphs may contain unreliable, noisy, or sub-optimal information, requiring researchers and practitioners to intentionally modify the graph topology and node features to improve downstream performance. Facing these two phenomena, our research works focus on natural and artificial dynamics for benefiting graph data mining and machine learning. In this paper, we will briefly introduce our recent works in investigative natural and artificial dynamics and point out some under-explored research problems in the interaction of these dynamics.
Cognitive-inspired Graph Redundancy Networks for Multi-source Information Fusion
Authors: Yao Fu, Junhong Wan, Junlan Yu, Weihao Jiang, Shiliang Pu
Venue: CIKM
Year: 2023

The recent developments in technologies bring not only increasing amount of information but also multiple information sources for Graph Representation Learning. With the success of Graph Neural Networks (GNN), there have been increasing attempts to learn representation of multi-source information leveraging its graph structures. However, existing graph methods basically combine multi-source information with different contribution scores and over-simplify the graph structures based on prior knowledge, which fail to unify complex and conflicting multi-source information. Multisensory Processing theory in cognitive neuroscience reveals human mechanism of learning multi-source information by identifying the redundancy and complementarity. Inspired by that, we propose Graph Redundancy Network (GRN) that: 1). learns a suitable representation space that maximizes multi-source interactions; 2). encodes the redundant and complementary information according to Graph Intersection and Difference of their graph structures; 3). further reinforces and explores the redundant and complementary information through low-pass and high-pass graph filters. The empirical study shows that GRN outperforms existing methods on various tasks.
Synergistic Disease Similarity Measurement via Unifying Hierarchical Relation Perception and Association Capturing
Authors: Zihao Gao, Huifang Ma, Yike Wang, Zhixin Li, Liang Chang
Venue: CIKM
Year: 2023

Quantifying similarities among human diseases is crucial to enhance our understanding of disease biology. Deep learning efforts have been devoted to quantifying disease similarity by integrating multi-view data sources from disparate biological data. However, disease data are often sparse, leading to suboptimal representation of disease given biological entity relationships and labeled disease data are not adequately modeled. In this paper, we propose an effective Synergistic disease Similarity measurement model called SynerSim. SynerSim possesses two key components: a hierarchical biological entity relation perception module to capture disease features from various biological entities, and a disease association capturing module based on signed random walk to model precious disease data. Additionally, SynerSim leverages dual granularity contrastive learning to enhance the representation of diverse biological entities, owing to the ability to enable the synergistic supervision of diseases represented by both homogeneous and heterogeneous information. Experimental results demonstrate that SynerSim achieves outstanding performance in the disease similarity measurement.
Predicting Interaction Quality of Conversational Assistants With Spoken Language Understanding Model Confidences
Authors: Yue Gao, Enrico Piovano, Tamer Soliman, Monir Moniruzzaman, Anoop Kumar, Melanie Bradford, Subhrangshu Nandi
Venue: CIKM
Year: 2023

In conversational AI assistants, SLU models are part of a complex pipeline composed of several modules working in harmony. Hence, an update to the SLU model needs to ensure improvements not only in the model specific metrics but also in the overall conversational assistant performance. Specifically, the impact on user interaction quality metrics must be factored in, while integrating interactions with distal modules upstream and downstream of the SLU component. We develop a ML model that makes it possible to gauge the interaction quality metrics due to SLU model changes before a production launch. The proposed model is a multi-modal transformer with a gated mechanism that conditions on text embeddings, output of a BERT model pre-trained on conversational data, and the hypotheses of the SLU classifiers with the corresponding confidence scores. We show that the proposed model predicts defect with more than 76% correlation with live interaction quality defects, compared to 46% baseline.
AI in the Gray: Exploring Moderation Policies in Dialogic Large Language Models vs. Human Answers in Controversial Topics
Authors: Vahid Ghafouri, Vibhor Agarwal, Yong Zhang, Nishanth Sastry, Jose M. Such, Guillermo Suarez-Tangil
Venue: CIKM
Year: 2023

The introduction of ChatGPT and the subsequent improvement of Large Language Models (LLMs) have prompted more and more individuals to turn to the use of ChatBots, both for information and assistance with decision-making. However, the information the user is after is often not formulated by these ChatBots objectively enough to be provided with a definite, globally accepted answer. Controversial topics, such as "religion", "gender identity", "freedom of speech", and "equality", among others, can be a source of conflict as partisan or biased answers can reinforce preconceived notions or promote disinformation. By exposing ChatGPT to such debatable questions, we aim to understand its level of awareness and if existing models are subject to socio-political and/or economic biases. We also aim to explore how AI-generated answers compare to human ones. For exploring this, we use a dataset of a social media platform created for the purpose of debating human-generated claims on polemic subjects among users, dubbed Kialo. Our results show that while previous versions of ChatGPT have had important issues with controversial topics, more recent versions of ChatGPT (gpt-3.5-turbo) are no longer manifesting significant explicit biases in several knowledge areas. In particular, it is well-moderated regarding economic aspects. However, it still maintains degrees of implicit libertarian leaning toward right-winged ideals which suggest the need for increased moderation from the socio-political point of view. In terms of domain knowledge on controversial topics, with the exception of the "Philosophical" category, ChatGPT is performing well in keeping up with the collective human level of knowledge. Finally, we see that sources of Bing AI have slightly more tendency to the center when compared to human answers. All the analyses we make are generalizable to other types of biases and domains.
Proactive Streaming Analytics at Scale: A Journey from the State-of-the-art to a Production Platform
Authors: Nikos Giatrakos, Elias Alevizos, Antonios Deligiannakis, Ralf Klinkenberg, Alexander Artikis
Venue: CIKM
Year: 2023

Proactive streaming analytics continuously extract real-time business value from massive data that stream in data centers or clouds. This requires (a) to process the data while they are still in motion; (b) to scale the processing to multiple machines, often over various, dispersed computer clusters, with diverse Big Data technologies; and (c) to forecast complex business events for proactive decision-making. Combining the necessary facilities for proactive streaming analytics at scale entails: (I) deep knowledge of the relevant state-of-the-art, (II) cherry-picking cutting edge research outcomes based on desired features and with the prospect of building interoperable components, and (III) building components and deploying them into a holistic architecture within a real-world platform. In this tutorial, we drive the audience through the whole journey from (I) to (III), delivering cutting edge research into a commercial analytics platform, for which we provide a hands-on experience.
DataDoc Analyzer: A Tool for Analyzing the Documentation of Scientific Datasets
Authors: Joan Giner-Miguelez, Abel Gómez, Jordi Cabot
Venue: CIKM
Year: 2023

Recent public regulatory initiatives and relevant voices in the ML community have identified the need to document datasets according to several dimensions to ensure the fairness and trustworthiness of machine learning systems. In this sense, the data-sharing practices in the scientific field have been quickly evolving in the last years, with more and more research works publishing technical documentation together with the data for replicability purposes. However, this documentation is written in natural language, and its structure, content focus, and composition vary, making them challenging to analyze. We present DataDoc Analyzer, a tool for analyzing the documentation of scientific datasets by extracting the details of the main dimensions required to analyze the fairness and potential biases. We believe that our tool could help improve the quality of scientific datasets, aid dataset curators during its documentation process, and be a helpful tool for empirical studies on the overall quality of the datasets used in the ML field. The tool implements an ML pipeline that uses Large Language Models at its core for information retrieval. DataDoc is open-source, and a public demo is published online.
Knowledge-Enhanced Multi-Label Few-Shot Product Attribute-Value Extraction
Authors: Jiaying Gong, Wei-Te Chen, Hoda Eldardiry
Venue: CIKM
Year: 2023

Existing attribute-value extraction (AVE) models require large quantities of labeled data for training. However, new products with new attribute-value pairs enter the market every day in real-world e-Commerce. Thus, we formulate AVE in multi-label few-shot learning (FSL), aiming to extract unseen attribute value pairs based on a small number of training examples. We propose a Knowledge-Enhanced Attentive Framework (KEAF) based on prototypical networks, leveraging the generated label description and category information to learn more discriminative prototypes. Besides, KEAF integrates with hybrid attention to reduce noise and capture more informative semantics for each class by calculating the label-relevant and query-related weights. To achieve multi-label inference, KEAF further learns a dynamic threshold by integrating the semantic information from both the support set and the query set. Extensive experiments with ablation studies conducted on two datasets demonstrate that our proposed model significantly outperforms other SOTA models for information extraction in few-shot learning.
Hierarchical Meta-Learning with Hyper-Tasks for Few-Shot Learning
Authors: Yunchuan Guan, Yu Liu, Ke Zhou, Junyuan Huang
Venue: CIKM
Year: 2023

Meta-learning excels in few-shot learning by extracting shared knowledge from the observed tasks. However, it needs the tasks to adhere to the i.i.d. constraint, which is challenging to achieve due to complex task relationships between data content. Current methods that create tasks in a one-dimensional structure and use meta-learning to learn all tasks flatly struggle with extracting shared knowledge from tasks with overlapping concepts. To address this issue, we propose further constructing tasks from the same environment into hyper-tasks. Since the distributions of hyper-tasks and tasks in a hyper-task can both be approximated as i.i.d. due to further summarization, the meta-learning algorithm can capture shared knowledge more efficiently. Based on the hyper-task, we propose a hierarchical meta-learning paradigm to meta-learn the meta-learning algorithm. The paradigm builds a customized meta-learner for each hyper-task, which makes meta-learners more flexible and expressive. We apply the paradigm to three classic meta-learning algorithms and conduct extensive experiments on public datasets, which confirm the superiority of hierarchical meta-learning in the few-shot learning setting. The code is released at https://github.com/tuantuange/H-meta-learning.
Explainable and Accurate Natural Language Understanding for Voice Assistants and Beyond
Authors: Kalpa Gunaratna, Vijay Srinivasan, Hongxia Jin
Venue: CIKM
Year: 2023

Joint intent detection and slot filling, which is also termed as joint NLU (Natural Language Understanding) is invaluable for smart voice assistants. Recent advancements in this area have been heavily focusing on improving accuracy using various techniques. Explainability is undoubtedly an important aspect for deep learning-based models including joint NLU models. Without explainability, their decisions are opaque to the outside world and hence, have tendency to lack user trust. Therefore to bridge this gap, we transform the full joint NLU model to be 'inherently' explainable at granular levels without compromising on accuracy. Further, as we enable the full joint NLU model explainable, we show that our extension can be successfully used in other general classification tasks. We demonstrate this using sentiment analysis and named entity recognition.
Leveraging Post-Click User Behaviors for Calibrated Conversion Rate Prediction Under Delayed Feedback in Online Advertising
Authors: Yuyao Guo, Xiang Ao, Qiming Liu, Qing He
Venue: CIKM
Year: 2023

Obtaining accurately calibrated conversion rate predictions is essential for the bidding and ranking process in online advertising systems. Nevertheless, the inherent latency between clicks and conversions leads to delayed feedback, which may introduce bias into the prediction models. Compared to indefinitely long conversion delays, post-click user behaviors manifest within a relatively brief time and have been empirically validated to exert a favorable influence on the precision of conversion rate estimates. In light of this, we propose a novel approach that leverages post-click user behaviors to calibrate conversion rate predictions. Specifically, we treat user behaviors as predictable targets to improve accuracy and enhance timeliness. An adaptive loss function based on task uncertainty is employed for multi-task learning. To further reduce calibration error, we integrate the modified prediction model with a parameterized scaling technique. Experiments conducted on two real-world datasets demonstrate that our proposed method outperforms existing models in providing more calibrated predictions.
MGICL: Multi-Grained Interaction Contrastive Learning for Multimodal Named Entity Recognition
Authors: Aibo Guo, Xiang Zhao, Zhen Tan, Weidong Xiao
Venue: CIKM
Year: 2023

Multimodal Named Entity Recognition (MNER) aims to combine data from different modalities (e.g. text, images, videos, etc.) for recognition and classification of named entities, which is crucial for constructing Multimodal Knowledge Graphs (MMKGs). However, existing researches suffer from two prominant issues: over-reliance on textual features while neglecting visual features, and the lack of effective reduction of the feature space discrepancy of multimodal data. To overcome these challenges, this paper proposes a Multi-Grained Interaction Contrastive Learning framework for MNER task, namely MGICL. MGICL slices data into different granularities, i.e., sentence level/word token level for text, and image level/object level for image. By utilizing multimodal features with different granularities, the framework enables cross-contrast and narrows down the feature space discrepancy between modalities. Moreover, it facilitates the acquisition of valuable visual features by the text. Additionally, a visual gate control mechanism is introduced to dynamically select relevant visual information, thereby reducing the impact of visual noise. Experimental results demonstrate that the proposed MGICL framework satisfactorily tackles the challenges of MNER through enhancing information interaction of multimodal data and reducing the effect of noise, and hence, effectively improves the performance of MNER.
Query-dominant User Interest Network for Large-Scale Search Ranking
Authors: Tong Guo, Xuanping Li, Haitao Yang, Xiao Liang, Yong Yuan, Jingyou Hou, Bingqing Ke, Chao Zhang, Junlin He, Shunyu Zhang, Enyun Yu, Wenwu Ou
Venue: CIKM
Year: 2023

Historical behaviors have shown great effect and potential in various prediction tasks, including recommendation and information retrieval. The overall historical behaviors are various but noisy while search behaviors are always sparse. Most existing approaches in personalized search ranking adopt the sparse search behaviors to learn representation with bottleneck, which do not sufficiently exploit the crucial long-term interest. In fact, there is no doubt that user long-term interest is various but noisy for instant search, and how to exploit it well still remains an open problem. To tackle this problem, in this work, we propose a novel model named Query-dominant user Interest Network (QIN), including two cascade units to filter the raw user behaviors and reweigh the behavior subsequences. Specifically, we propose a relevance search unit (RSU), which aims to search a subsequence relevant to the query first and then search the sub-subsequences relevant to the target item. These items are then fed into an attention unit called Fused Attention Unit (FAU). It should be able to calculate attention scores from the ID field and attribute field separately, and then adaptively fuse the item embedding and content embedding based on the user engagement of past period. Extensive experiments and ablation studies on real-world datasets demonstrate the superiority of our model over state-of-the-art methods. The QIN now has been successfully deployed on Kuaishou search, an online video search platform, and obtained 7.6% improvement on CTR.
Interpretable Fake News Detection with Graph Evidence
Authors: Hao Guo, Weixin Zeng, Jiuyang Tang, Xiang Zhao
Venue: CIKM
Year: 2023

Automatic detection of fake news has received widespread attentions over recent years. A pile of efforts has been put forward to address the problem with high accuracy, while most of them lack convincing explanations, making it difficult to curb the continued spread of false news in real-life cases. Although some models leverage external resources to provide preliminary interpretability, such external signals are not always available. To fill in this gap, in this work, we put forward an interpretable fake news detection model IKA by making use of the historical evidence in the form of graphs. Specifically, we establish both positive and negative evidence graphs by collecting the signals from the historical news, i.e., training data. Then, given a piece of news to be detected, in addition to the common features used for detecting false news, we compare the news and evidence graphs to generate both the matching vector and the related graph evidence for explaining the prediction. We conduct extensive experiments on both Chinese and English datasets. The experiment results show that the detection accuracy of IKA exceeds the state-of-the-art approaches and IKA can provide useful explanations for the prediction results. Besides, IKA is general and can be applied on other models to improve their interpretability.
Hateful Comment Detection and Hate Target Type Prediction for Video Comments
Authors: Shrey Gupta, Pratyush Priyadarshi, Manish Gupta
Venue: CIKM
Year: 2023

With the widespread increase in hateful content on the web, hate detection has become more crucial than ever. Although vast literature exists on hate detection from text, images and videos, interestingly, there has been no previous work on hateful comment detection (HCD) from video pages. HCD is critical for comment moderation and for flagging controversial videos. Comments are often short, contextual and convoluted making the problem challenging. Toward solving this problem, we contribute a dataset, HateComments, consisting of 2071 comments for 401 videos obtained from two popular video sharing platforms. We investigate two related tasks: binary HCD and 4-class multi-label hate target-type prediction (HTP). We systematically explore the importance of various forms of context for effective HCD. Our initial experiments show that our best method which leverages rich video context (like description, transcript and visual input) leads to an HCD accuracy of ~78.6% and an ROC AUC score of ~0.61 for HTP. Code and data is at https://drive.google.com/file/d/1EUbWDUokv1CYkWKlwByUC6yIuBGUw2MN/.
Workshop on Enterprise Knowledge Graphs using Large Language Models
Authors: Rajeev Gupta, Srinath Srinivasa
Venue: CIKM
Year: 2023

Knowledge graphs are used for organizing and connecting individual entities to integrate the information extracted from different data sources. Typically, knowledge graphs are used to connect various real-world entities like persons, places, things, actions, etc. For the knowledge graphs created using the enterprise data, the knowledge graph entities can be of different types-static entities (e.g., people, projects), communication entities (e.g., emails, meetings, documents), derived entities (e.g., rules, definitions, entities from emails), etc. The graphs are used to connect these entities with enriched context (as edges and node attributes) and used for powering various search and recommendations applications. With the advent of large language models, the whole lifecycle of knowledge graphs involving -information extraction, graph construction, application of graphs, querying knowledge graphs, using the graph for recommendations, etc., - is impacted. With large language models such as GPT, LLaMA, PALM, etc., entity and relationship extraction can be improved. Similarly, one can answer different types of queries with the help of LLMs which were very difficult without them. This workshop is about improving the enterprise knowledge graphs and its applications using large language models. Enterprise graphs can be of different scopes-whether they contain data from individual users/customers, a sub-organization, or the whole enterprise. This workshop will also cover various privacy and access control related issues which are typical for any enterprise graph. These include privacy preserving federated learning, using LLMs to extract information from private data, querying the knowledge graph in a privacy preserving manner, etc.