GenAI4ELab/papercli-papers huggingface dataset:
Graph Neural Networks for Brain Graph Learning: A Survey
Authors: Xuexiong Luo; Jia Wu; Jian Yang; Shan Xue; Amin Beheshti; Quan Z. Sheng; David McAlpine; Paul Sowman; Alexis Giral; Philip S. Yu
Venue: IJCAI 2024 (main)

Abstract: Exploring the complex structure of the human brain is crucial for understanding its functionality and diagnosing brain disorders. Thanks to advancements in neuroimaging technology, a novel approach has emerged that involves modeling the human brain as a graph-structured pattern, with different brain regions represented as nodes and the functional relationships among these regions as edges. Moreover, graph neural networks (GNNs) have demonstrated a significant advantage in mining graph-structured data. Developing GNNs to learn brain graph representations for brain disorder analysis has recently gained increasing attention. However, there is a lack of systematic survey work summarizing current research methods in this domain. In this paper, we aim to bridge this gap by reviewing brain graph learning works that utilize GNNs. We first introduce the process of brain graph modeling based on common neuroimaging data. Subsequently, we systematically categorize current works based on the type of brain graph generated and the targeted research problems. To make this research accessible to a broader range of interested researchers, we provide an overview of representative methods and commonly used datasets, along with their implementation sources. Finally, we present our insights on future research directions. The repository of this survey is available at https://github.com/XuexiongLuoMQ/Awesome-Brain-Graph-Learning-with-GNNs.
X-former Elucidator: Reviving Efficient Attention for Long Context Language Modeling
Authors: Xupeng Miao; Shenhan Zhu; Fangcheng Fu; Ziyu Guo; Zhi Yang; Yaofeng Tu; Zhihao Jia; Bin Cui
Venue: IJCAI 2024 (main)

Abstract: Transformer-based LLMs are becoming increasingly important in various AI applications. However, apart from the success of LLMs, the explosive demand of long context handling capabilities is a key and in-time problem for both academia and industry. Due to the limitations from the quadratic complexity of the attention mechanism, long context scenarios require much more resources for LLM development and deployment, bringing huge challenges to the underlying AI infrastructure. Meanwhile, we observe that there is a trend of reviving previous efficient attention mechanisms to latest LLMs. However, it still remains an open question about how to select from these diverse approaches in practice. In this paper, we answer this question from several aspects. First, we revisit these latest long-context LLM innovations and discuss their relationship with prior approaches with a novel and comprehensive taxonomy. Next, we conduct a thorough evaluation over various types of workloads considering both efficiency and effectiveness. Finally, we provide an in-depth analysis, summarize our key findings, and offer insightful suggestions on the trade-offs of designing and deploying efficient attention mechanisms for Transformer-based LLMs.
Knowledge Distillation in Federated Learning: A Practical Guide
Authors: Alessio Mora; Irene Tenison; Paolo Bellavista; Irina Rish
Venue: IJCAI 2024 (main)

Abstract: Federated Learning (FL) enables the training of Deep Learning models without centrally collecting possibly sensitive raw data. The most used algorithms for FL are parameter-averaging based schemes (e.g., Federated Averaging) that, however, have well known limits, i.e., model homogeneity, high communication cost, poor performance in presence of heterogeneous data distributions. Federated adaptations of regular Knowledge Distillation (KD) can solve or mitigate the weaknesses of parameter-averaging FL algorithms while possibly introducing other trade-offs. In this article, we originally present a focused review of the state-of-the-art KD-based algorithms specifically tailored for FL, by providing both a novel classification of the existing approaches and a detailed technical description of their pros, cons, and tradeoffs.
A Comprehensive Survey of Cross-Domain Policy Transfer for Embodied Agents
Authors: Haoyi Niu; Jianming Hu; Guyue Zhou; Xianyuan Zhan
Venue: IJCAI 2024 (main)

Abstract: The burgeoning fields of robot learning and embodied AI have triggered an increasing demand for large quantities of data. However, collecting sufficient unbiased data from the target domain remains a challenge due to costly data collection processes and stringent safety requirements. Consequently, researchers often resort to data from easily accessible source domains, such as simulation and laboratory environments, for cost-effective data acquisition and rapid model iteration. Nevertheless, the environments and embodiments of these source domains can be quite different from their target domain counterparts, underscoring the need for effective cross-domain policy transfer approaches. In this paper, we conduct a systematic review of existing cross-domain policy transfer methods. Through a nuanced categorization of domain gaps, we encapsulate the overarching insights and design considerations of each problem setting. We also provide a high-level discussion about the key methodologies used in cross-domain policy transfer problems. Lastly, we summarize the open challenges that lie beyond the capabilities of current paradigms and discuss potential future directions in this field.
Learning Structural Causal Models through Deep Generative Models: Methods, Guarantees, and Challenges
Authors: Audrey Poinsot; Alessandro Leite; Nicolas Chesneau; Michele Sebag; Marc Schoenauer
Venue: IJCAI 2024 (main)

Abstract: This paper provides a comprehensive review of deep structural causal models (DSCMs), particularly focusing on their ability to answer counterfactual queries using observational data within known causal structures. It delves into the characteristics of DSCMs by analyzing the hypotheses, guarantees, and applications inherent to the underlying deep learning components and structural causal models, fostering a finer understanding of their capabilities and limitations in addressing different counterfactual queries. Furthermore, it highlights the challenges and open questions in the field of deep structural causal modeling. It sets the stages for researchers to identify future work directions and for practitioners to get an overview in order to find out the most appropriate methods for their needs.
A Survey on Network Alignment: Approaches, Applications and Future Directions
Authors: Shruti Saxena; Joydeep Chandra
Venue: IJCAI 2024 (main)

Abstract: Network alignment, the task of mapping corresponding nodes across networks, is attracting more attention for cross-network analysis in diverse domains, including social, biological, and co-authorship networks. Although a variety of methods have been proposed, we lack a holistic understanding of the approaches and applications. Our survey aims to bridge this gap by first proposing a taxonomy of network alignment, characterizing existing approaches, and then systematically summarizing and reviewing their performance and highlighting their scopes for future development. Finally, we discuss some important applications and give directions for future research within this domain.
Supervised Algorithmic Fairness in Distribution Shifts: A Survey
Authors: Minglai Shao; Dong Li; Chen Zhao; Xintao Wu; Yujie Lin; Qin Tian
Venue: IJCAI 2024 (main)

Abstract: Supervised fairness-aware machine learning under distribution shifts is an emerging field that addresses the challenge of maintaining equitable and unbiased predictions when faced with changes in data distributions from source to target domains. In real-world applications, machine learning models are often trained on a specific dataset but deployed in environments where the data distribution may shift over time due to various factors. This shift can lead to unfair predictions, disproportionately affecting certain groups characterized by sensitive attributes, such as race and gender. In this survey, we provide a summary of various types of distribution shifts and comprehensively investigate existing methods based on these shifts, highlighting six commonly used approaches in the literature. Additionally, this survey lists publicly available datasets and evaluation metrics for empirical studies. We further explore the interconnection with related research fields, discuss the significant challenges, and identify potential directions for future studies.
Building Expressive and Tractable Probabilistic Generative Models: A Review
Authors: Sahil Sidheekh; Sriraam Natarajan
Venue: IJCAI 2024 (main)

Abstract: We present a comprehensive survey of the advancements and techniques in the field of tractable probabilistic generative modeling, primarily focusing on Probabilistic Circuits (PCs). We provide a unified perspective on the inherent trade-offs between expressivity and tractability, highlighting the design principles and algorithmic extensions that have enabled building expressive and efficient PCs, and provide a taxonomy of the field. We also discuss recent efforts to build deep and hybrid PCs by fusing notions from deep neural models, and outline the challenges and open questions that can guide future research in this evolving field.
A Systematic Survey on Federated Semi-supervised Learning
Authors: Zixing Song; Xiangli Yang; Yifei Zhang; Xinyu Fu; Zenglin Xu; Irwin King
Venue: IJCAI 2024 (main)

Abstract: Federated learning (FL) revolutionizes distributed machine learning by enabling devices to collaboratively learn a model while maintaining data privacy. However, FL usually faces a critical challenge with limited labeled data, making semi-supervised learning (SSL) crucial for utilizing abundant unlabeled data. The integration of SSL within the federated framework gives rise to federated semi-supervised learning (FSSL), a novel approach that exploits unlabeled data across devices without compromising privacy. This paper systematically explores FSSL, shedding light on its four basic problem settings that commonly appear in real-world scenarios. By examining the unique challenges, generic solutions, and representative methods tailored for each setting of FSSL, we aim to provide a cohesive overview of the current state of the art and pave the way for future research directions in this promising field.
Intelligent Agents for Auction-based Federated Learning: A Survey
Authors: Xiaoli Tang; Han Yu; Xiaoxiao Li; Sarit Kraus
Venue: IJCAI 2024 (main)

Abstract: Auction-based federated learning (AFL) is an important emerging category of FL incentive mechanism design, due to its ability to fairly and efficiently motivate high-quality data owners to join data consumers' (i.e., servers') FL training tasks. To enhance the efficiency in AFL decision support for stakeholders (i.e., data consumers, data owners, and the auctioneer), intelligent agent-based techniques have emerged. However, due to the highly interdisciplinary nature of this field and the lack of a comprehensive survey providing an accessible perspective, it is a challenge for researchers to enter and contribute to this field. This paper bridges this important gap by providing a first-of-its-kind survey on the Intelligent Agents for AFL (IA-AFL) literature. We propose a unique multi-tiered taxonomy that organises existing IA-AFL works according to 1) the stakeholders served, 2) the auction mechanism adopted, and 3) the goals of the agents, to provide readers with a multi-perspective view into this field. In addition, we analyse the limitations of existing approaches, summarise the commonly adopted performance evaluation metrics, and discuss promising future directions leading towards effective and efficient stakeholder-oriented decision support in IA-AFL ecosystems.
A Survey of Constraint Formulations in Safe Reinforcement Learning
Authors: Akifumi Wachi; Xun Shen; Yanan Sui
Venue: IJCAI 2024 (main)

Abstract: Safety is critical when applying reinforcement learning (RL) to real-world problems. As a result, safe RL has emerged as a fundamental and powerful paradigm for optimizing an agent’s policy while incorporating notions of safety. A prevalent safe RL approach is based on a constrained criterion, which seeks to maximize the expected cumulative reward subject to specific safety constraints. Despite recent effort to enhance safety in RL, a systematic understanding of the field remains difficult. This challenge stems from the diversity of constraint representations and little exploration of their interrelations. To bridge this knowledge gap, we present a comprehensive review of representative constraint formulations, along with a curated selection of algorithms designed specifically for each formulation. In addition, we elucidate the theoretical underpinnings that reveal the mathematical mutual relations among common problem formulations. We conclude with a discussion of the current state and future directions of safe reinforcement learning research
Recent Advances in Predictive Modeling with Electronic Health Records
Authors: Jiaqi Wang; Junyu Luo; Muchao Ye; Xiaochen Wang; Yuan Zhong; Aofei Chang; Guanjie Huang; Ziyi Yin; Cao Xiao; Jimeng Sun; Fenglong Ma
Venue: IJCAI 2024 (main)

Abstract: The development of electronic health records (EHR) systems has enabled the collection of a vast amount of digitized patient data. However, utilizing EHR data for predictive modeling presents several challenges due to its unique characteristics. With the advancements in machine learning techniques, deep learning has demonstrated its superiority in various applications, including healthcare. This survey systematically reviews recent advances in deep learning-based predictive models using EHR data. Specifically, we introduce the background of EHR data and provide a mathematical definition of the predictive modeling task. We then categorize and summarize predictive deep models from multiple perspectives. Furthermore, we present benchmarks and toolkits relevant to predictive modeling in healthcare. Finally, we conclude this survey by discussing open challenges and suggesting promising directions for future research.
A Survey on Rank Aggregation
Authors: Siyi Wang; Qi Deng; Shiwei Feng; Hong Zhang; Chao Liang
Venue: IJCAI 2024 (main)

Abstract: Rank aggregation (RA), the technique of combining multiple basic rankings into a consensus one, plays an important role in social choices, bioinformatics, information retrieval, metasearch, and recommendation systems. Although recent years have witnessed remarkable progress in RA, the absence of a systematic overview motivates us to conduct a comprehensive survey including both classic algorithms and the latest advances in RA study. Specifically, we first discuss the challenges of RA research, then present a systematic review with a fine-grained taxonomy to introduce representative algorithms in unsupervised RA, supervised RA, as well as the previously overlooked semi-supervised RA. Within each category, we not only summarize the common ideas of similar methods, but also discuss their strengths and weaknesses. Particularly, to investigate the performance difference of different types of RA methods, we conduct the largest scale of comparative evaluation to date of 27 RA methods on 7 public datasets from person re-identification, recommendation systems, bioinformatics and social choices. Finally, we raise two open questions in the current RA research and make our comments about future trends in the context of the latest research progress.
A Survey on Extractive Knowledge Graph Summarization: Applications, Approaches, Evaluation, and Future Directions
Authors: Xiaxia Wang; Gong Cheng
Venue: IJCAI 2024 (main)

Abstract: With the continuous growth of large Knowledge Graphs (KGs), extractive KG summarization becomes a trending task. Aiming at distilling a compact subgraph with condensed information, it facilitates various downstream KG-based tasks. In this survey paper, we are among the first to provide a systematic overview of its applications and define a taxonomy for existing methods from its interdisciplinary studies. Future directions are also laid out based on our extensive and comparative review.
Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models
Authors: Xindi Wang; Mahsa Salmani; Parsa Omidi; Xiangyu Ren; Mehdi Rezagholizadeh; Armaghan Eshaghi
Venue: IJCAI 2024 (main)

Abstract: Recently, large language models (LLMs) have shown remarkable capabilities including understanding context, engaging in logical reasoning, and generating responses. However, this is achieved at the expense of stringent computational and memory requirements, hindering their ability to effectively support long input sequences. This survey provides an inclusive review of the recent techniques and methods devised to extend the sequence length in LLMs, thereby enhancing their capacity for long-context understanding. In particular, we review and categorize a wide range of techniques including architectural modifications, such as modified positional encoding and altered attention mechanisms, which are designed to enhance the processing of longer sequences while avoiding a proportional increase in computational cost. The diverse methodologies investigated in this study can be leveraged across different phases of LLMs, i.e., training, fine-tuning and inference. This enables LLMs to efficiently process extended sequences. The limitations of the current methodologies is discussed in the last section along with the suggestions for future research directions, underscoring the importance of sequence length in the continued advancement of LLMs.
On the Essence and Prospect: An Investigation of Alignment Approaches for Big Models
Authors: Xinpeng Wang; Shitong Duan; Xiaoyuan Yi; Jing Yao; Shanlin Zhou; Zhihua Wei; Peng Zhang; Dongkuan Xu; Maosong Sun; Xing Xie
Venue: IJCAI 2024 (main)

Abstract: Big models have achieved revolutionary breakthroughs in the field of AI, but they also pose potential ethical and societal risks to humans. Addressing such problems, alignment technologies were introduced to make these models conform to human preferences and values. Despite the considerable advancements in the past year, various challenges lie in establishing the optimal alignment strategy, such as data cost and scalable oversight, and how to align remains an open question. In this survey paper, we comprehensively investigate value alignment approaches. We first unpack the historical context of alignment tracing back to the 1920s (where it comes from), then delve into the mathematical essence of alignment (what it is), shedding light on the inherent challenges. Following this foundation, we provide a detailed examination of existing alignment methods, which fall into three categories: RL-based Alignment, SFT-based Alignment, and Inference-Time Alignment, and demonstrate their intrinsic connections, strengths, and limitations, helping readers better understand this research area. In addition, two emerging topics, alignment goal and multimodal alignment, are also discussed as novel frontiers in the field. Looking forward, we discuss potential alignment paradigms and how they could handle remaining challenges, prospecting where future alignment will go.
A Survey on Efficient Federated Learning Methods for Foundation Model Training
Authors: Herbert Woisetschläger; Alexander Erben; Shiqiang Wang; Ruben Mayer; Hans-Arno Jacobsen
Venue: IJCAI 2024 (main)

Abstract: Federated Learning (FL) has become an established technique to facilitate privacy-preserving collaborative training across a multitude of clients. However, new approaches to FL often discuss their contributions involving small deep-learning models only and focus on training full models on clients. In the wake of Foundation Models (FM), the reality is different for many deep learning applications. Typically, FMs have already been pre-trained across a wide variety of tasks and can be fine-tuned to specific downstream tasks over significantly smaller datasets than required for full model training. However, access to such datasets is often challenging. By its design, FL can help to open data silos. With this survey, we introduce a novel taxonomy focused on computational and communication efficiency, the vital elements to make use of FMs in FL systems. We discuss the benefits and drawbacks of parameter-efficient fine-tuning (PEFT) for FL applications, elaborate on the readiness of FL frameworks to work with FMs and provide future research opportunities on how to evaluate generative models in FL as well as the interplay of privacy and PEFT.
AI-Enhanced Virtual Reality in Medicine: A Comprehensive Survey
Authors: Yixuan Wu; Kaiyuan Hu; Danny Z. Chen; Jian Wu
Venue: IJCAI 2024 (main)

Abstract: With the rapid advance of computer graphics and artificial intelligence technologies, the ways we interact with the world have undergone a transformative shift. Virtual Reality (VR) technology, aided by artificial intelligence (AI), has emerged as a dominant interaction media in multiple application areas, thanks to its advantage of providing users with immersive experiences. Among those applications, medicine is considered one of the most promising areas. In this paper, we present a comprehensive examination of the burgeoning field of AI-enhanced VR applications in medical care and services. By introducing a systematic taxonomy, we meticulously classify the pertinent techniques and applications into three well-defined categories based on different phases of medical diagnosis and treatment: Visualization Enhancement, VR-related Medical Data Processing, and VR-assisted Intervention. This categorization enables a structured exploration of the diverse roles that AI-powered VR plays in the medical domain, providing a framework for a more comprehensive understanding and evaluation of these technologies.nTo our best knowledge, this work is the first systematic survey of AI-powered VR systems in medical settings, laying a foundation for future research in this interdisciplinary domain.
Large Language Models for Time Series: A Survey
Authors: Xiyuan Zhang; Ranak Roy Chowdhury; Rajesh K. Gupta; Jingbo Shang
Venue: IJCAI 2024 (main)

Abstract: Large Language Models (LLMs) have seen significant use in domains such as natural language processing and computer vision. Going beyond text, image and graphics, LLMs present a significant potential for analysis of time series data, benefiting domains such as climate, IoT, healthcare, traffic, audio and finance. This survey paper provides an in-depth exploration and a detailed taxonomy of the various methodologies employed to harness the power of LLMs for time series analysis. We address the inherent challenge of bridging the gap between LLMs' original text data training and the numerical nature of time series data, and explore strategies for transferring and distilling knowledge from LLMs to numerical time series analysis. We detail various methodologies, including (1) direct prompting of LLMs, (2) time series quantization, (3) aligning techniques, (4) utilization of the vision modality as a bridging mechanism, and (5) the combination of LLMs with tools. Additionally, this survey offers a comprehensive overview of the existing multimodal time series and text datasets in diverse domains, and discusses the challenges and future opportunities of this emerging field.