GenAI4ELab/papercli-papers huggingface dataset:
Knowledge-Enhanced Hierarchical Heterogeneous Graph for Personality Identification with Limited Training Data
Authors: Yuxuan Song; Qiudan Li; Yilin Wu; David Jingjun Xu; Daniel Dajun Zeng
Venue: AAAI 2025 (main)

Abstract: Personality identification plays important roles in understanding user behavior and offering foresight ability for downstream applications. The key challenge is how to address the scarcity of labeled personality data. Recently, some studies have adopted data augmentation and prompt learning to perform personality identification. However, they still heavily require a large amount of labeled data to learn an appropriate distance strategy, which limits the generalization and flexibility of the model. This study proposes a knowledge-enhanced hierarchical heterogeneous graph model, which adopts a global multi-view graph node encoding to acquire comprehensive personality features and their inherent associations, where  three types of knowledge including part-of-speech (POS) tag, entity, and Linguistic Inquiry and Word Count (LIWC) are introduced. Then, a hierarchical heterogeneous graph with a “post-word-diverse knowledge” structure is constructed for each post to obtain enhanced representation. Finally, a relation guided representation optimization that considers intra-user relationships and inter-label relationships is further developed to learn more discriminative semantic representation. Experimental results on three widely used datasets demonstrate that the model outperforms state-of-the-art methods when training with only 100 samples (approximately 1% of the total data set).
Pose as a Modality: A Psychology-Inspired Network for Personality Recognition with a New Multimodal Dataset
Authors: Bin Tang; Ke-Qi Pan; Miao Zheng; Ning Zhou; Jia-Lu Sui; Dandan Zhu; Cheng-Long Deng; Shu-Guang Kuai
Venue: AAAI 2025 (main)

Abstract: In recent years, predicting Big Five personality traits from multimodal data has received significant attention in artificial intelligence (AI). However, existing computational models often fail to achieve satisfactory performance. Psychological research has shown a strong correlation between pose and personality traits, yet previous research has largely ignored pose data in computational models. To address this gap, we develop a novel multimodal dataset that incorporates full-body pose data. The dataset includes video recordings of 287 participants completing a virtual interview with 36 questions, along with self-reported Big Five personality scores as labels. To effectively utilize this multimodal data, we introduce the Psychology-Inspired Network (PINet), which consists of three key modules: Multimodal Feature Awareness (MFA), Multimodal Feature Interaction (MFI), and Psychology-Informed Modality Correlation Loss (PIMC Loss). The MFA module leverages the Vision Mamba Block to capture comprehensive visual features related to personality, while the MFI module efficiently fuses the multimodal features. The PIMC Loss, grounded in psychological theory, guides the model to emphasize different modalities for different personality dimensions. Experimental results show that the PINet outperforms several state-of-the-art baseline models. Furthermore, the three modules of PINet contribute almost equally to the model’s overall performance. Incorporating pose data significantly enhances the model’s performance, with the pose modality ranking mid-level in importance among the five modalities. These findings address the existing gap in personality-related datasets that lack full-body pose data and provide a new approach for improving the accuracy of personality prediction models, highlighting the importance of integrating psychological insights into AI frameworks.
A Multi-Focus-Driven Multi-Branch Network for Robust Multimodal Sentiment Analysis
Authors: Chuanqi Tao; Jiaming Li; Tianzi Zang; Peng Gao
Venue: AAAI 2025 (main)

Abstract: Multimodal sentiment analysis aims to integrate diverse modalities for precise emotional interpretation. However, external factors such as sensor malfunctions or network issues may disrupt certain modalities. This may lead to missing data, which poses challenges in real-world deployment. Most existing approaches focus on designing feature reconstruction strategies, overlooking the collaborative integration of reconstruction and fusion strategies. Moreover, they fail to capture the relationships between features in the global dimension and those in the local dimension. These limitations hinder the full capture of the complex nature of multimodal data, especially in scenarios involving missing modalities. To address the above issues, this paper proposes a robust model named MFMB-Net with multiple branches for feature multi-focus fusion and reconstruction. We design a two-stream fusion branch where macro-fusion focuses on the fusion of features in the global dimension and micro-fusion targets local dimension features. This dual-stream fusion branch distributes multi-focus across both pathways, simultaneously capturing global coarse-grained and local fine-grained features. Additionally, the reconstruction branch interacts collaboratively with the fusion branch to reconstruct and enhance the missing data. It integrates the reconstructed feature information with the fused information thus refining the representation fidelity of the missing information. Experiments performed on two benchmarks show that our approach obtains results superior to state-of-the-art models.
Alignment of CNN and Human Judgments of Geometric and Topological Concepts
Authors: Neha Upadhyay; Vijay Marupudi; Kamala Varma; Sashank Varma
Venue: AAAI 2025 (main)

Abstract: AI and ML are poised to provide new insights into mathematical cognition and development. Here, we focus on the domains of geometry and topology (GT). According to one prominent developmental perspective, infants possess core knowledge of GT concepts, presumably underwritten by dedicated neural circuitry. We use the alignment between human cognition and computer vision models to evaluate an alternate proposal: that these concepts are learned “for free” through experience with the visual world. Specifically, we measure the sensitivity of five convolutional neural network (CNN) models to 43 GT concepts that aggregate into seven classes. We focus on CNNs over other architectures (e.g., vision transformers) because their neural plausibility has been established through studies mapping their layers to areas of the brain’s ventral visual stream. We find evidence that the CNNs are sensitive to some classes (e.g., Euclidean Geometry) but not others (e.g., Geometric Transformations). The models’ sensitivity is generally lower at lower layers and maximal at the final fully-connected layer. Experiments with models from the ResNet family show that increasing model depth does not necessarily increase sensitivity to GT concepts. The models’ profiles of sensitivity to the seven classes roughly align with the profile shown by humans, with ResNet-18 corresponding best to Western adults and DenseNet to Western children ages 3-6 years. This case study shows how CNNs can provide sufficiency proofs for the learnability of mathematical concepts and thus inform theoretical debates in cognitive and developmental science. These findings set the stage for future experiments with other vision model architectures.
DDJND: Dual Domain Just Noticeable Difference in Multi-Source Content Images with Structural Discrepancy
Authors: Miaohui Wang; Zhenming Li; Wuyuan Xie
Venue: AAAI 2025 (main)

Abstract: Most existing just noticeable difference (JND) methods primarily integrate specific masking effects in a single domain. However, these single-domain JND methods struggle with the structural discrepancies in multi-source content images, limiting their effectiveness in visual redundancy estimation. To address this  issue, we propose a dual domain encoder that combines spatial and frequency features to comprehensively capture visual patterns. Our design includes spatial pattern balance and frequency detail correction modules to balance global and local patterns and correct low- and high-frequency  distributions. Additionally, we develop a dual domain decoder to effectively extract multi-scale pattern redundancies and integrate them with detail redundancies in the frequency domain. Experiments demonstrate the effectiveness and robustness of our proposed method in handling structural discrepancies in multi-source content images.
BIG-FUSION: Brain-Inspired Global-Local Context Fusion Framework for Multimodal Emotion Recognition in Conversations
Authors: Yusong Wang; Xuanye Fang; Huifeng Yin; Dongyuan Li; Guoqi Li; Qi Xu; Yi Xu; Shuai Zhong; Mingkun Xu
Venue: AAAI 2025 (main)

Abstract: Considering the importance of capturing both global conversational topics and local speaker dependencies for multimodal emotion recognition in conversations, current approaches first utilize sequence models like Transformer to extract global context information, then apply Graph Neural Networks to model local speaker dependencies for local context information extraction, coupled with Graph Contrastive Learning (GCL) to enhance node representation learning. However, this sequential design introduces potential biases: the extracted global context information inevitably influences subsequent processing, compromising the independence and diversity of the original local features; current graph augmentation methods in GCL cannot consider both global and local context information in conversations to evaluate the node importance, hindering the learning of key information. Inspired by the human brain excels at handling complex tasks by efficiently integrating local and global information processing mechanisms, we propose an aligned global-local context fusion framework for sequence-based design to address these problems. This design includes a dual-attention Transformer and a dual-evaluation method for graph augmentation in GCL. The dual-attention Transformer combines global attention for overall context extraction with sliding-window attention for local context capture, both enhanced by spiking neuron dynamics. The dual-evaluation method in GCL comprises global importance evaluation to identify nodes crucial for overall conversation context, and local importance evaluation to detect nodes significant for local semantics, generating augmented graph views that preserve both global and local information. This approach ensures balanced information processing throughout the pipeline, enhancing biological plausibility and achieving superior emotion recognition.
Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Networks
Authors: Ziqing Wang; Yuetong Fang; Jiahang Cao; Hongwei Ren; Renjing Xu
Venue: AAAI 2025 (main)

Abstract: Spiking Neural Networks (SNNs) are seen as an energy-efficient alternative to traditional Artificial Neural Networks (ANNs), but the performance gap remains a challenge. While this gap is narrowing through ANN-to-SNN conversion, substantial computational resources are still needed, and the energy efficiency of converted SNNs cannot be ensured. To address this, we present a unified training-free conversion framework that significantly enhances both the performance and efficiency of converted SNNs. Inspired by the biological nervous system, we propose a novel Adaptive-Firing Neuron Model (AdaFire), which dynamically adjusts firing patterns across different layers to substantially reduce the Unevenness Error - the primary source of error of converted SNNs within limited inference timesteps. We further introduce two efficiency-enhancing techniques: the Sensitivity Spike Compression (SSC) technique for reducing spike operations, and the Input-aware Adaptive Timesteps (IAT) technique for decreasing latency. These methods collectively enable our approach to achieve state-of-the-art performance with significant energy savings of up to 70.1%, 60.3%, and 43.1% on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively. Extensive experiments across 2D, 3D, event-driven classification tasks, object detection, and segmentation tasks, demonstrate the effectiveness of our method in various domains.
Undermining Mental Proof: How AI Can Make Cooperation Harder by Making Thinking Easier
Authors: Zachary Wojtowicz; Simon DeDeo
Venue: AAAI 2025 (main)

Abstract: Large language models and other highly capable AI systems ease the burdens of deciding what to say or do, but this very ease can undermine the effectiveness of our actions in social contexts. We explain this apparent tension by introducing the integrative theoretical concept of "mental proof," which occurs when observable actions are used to certify unobservable mental facts. From hiring to dating, mental proofs enable people to credibly communicate values, intentions, states of knowledge, and other private features of their minds to one another in low-trust environments where honesty cannot be easily enforced. Drawing on results from economics, theoretical biology, and computer science, we describe the core theoretical mechanisms that enable people to effect mental proofs. An analysis of these mechanisms clarifies when and how artificial intelligence can make low-trust cooperation harder despite making thinking easier.
Enriching Multimodal Sentiment Analysis Through Textual Emotional Descriptions of Visual-Audio Content
Authors: Sheng Wu; Dongxiao He; Xiaobao Wang; Longbiao Wang; Jianwu Dang
Venue: AAAI 2025 (main)

Abstract: Multimodal Sentiment Analysis (MSA) stands as a critical research frontier, seeking to comprehensively unravel human emotions by amalgamating text, audio, and visual data. Yet, discerning subtle emotional nuances within audio and video expressions poses a formidable challenge, particularly when emotional polarities across various segments appear similar. In this paper, our objective is to spotlight emotion-relevant attributes of audio and visual modalities to facilitate multimodal fusion in the context of nuanced emotional shifts in visual-audio scenarios. To this end, we introduce DEVA, a progressive fusion framework founded on textual sentiment descriptions aimed at accentuating emotional features of visual-audio content. DEVA employs an Emotional Description Generator (EDG) to transmute raw audio and visual data into textualized sentiment descriptions, thereby amplifying their emotional characteristics. These descriptions are then integrated with the source data to yield richer, enhanced features. Furthermore, DEVA incorporates the Text-guided Progressive Fusion Module (TPF), leveraging varying levels of text as a core modality guide. This module progressively fuses visual-audio minor modalities to alleviate disparities between text and visual-audio modalities. Experimental results on widely used sentiment analysis benchmark datasets, including MOSI, MOSEI, and CH-SIMS, underscore significant enhancements compared to state-of-the-art models. Moreover, fine-grained emotion experiments corroborate the robust sensitivity of DEVA to subtle emotional variations.
DepMGNN: Matrixial Graph Neural Network for Video-based Automatic Depression Assessment
Authors: Zijian Wu; Leijing Zhou; Shuanglin Li; Changzeng Fu; Jun Lu; Jing Han; Yi Zhang; Zhuang Zhao; Siyang Song
Venue: AAAI 2025 (main)

Abstract: Depression can be reflected by long-term human spatio-temporal facial behaviours. While human face videos recorded in real-world usually have long and variable lengths, existing video-based depression assessment approaches frequently re-sample/down-sample such videos to short and equal-length videos, or split each video into several equal-length segments, where segment-level spatio-temporal facial behaviours are suppressed as a vector-style representations for RNN-based long-term (video-level) modelling. Both strategies lead to crucial information loss and distortion. In this paper, we propose a novel graph-style data structure called Matrixial Graph and an effective Matrixial Graph Neural Network (MGNN) for face video-based depression assessment, which can directly and end-to-end model long-term depression-specific spatio-temporal facial cues from variable-length videos without resampling/splitting videos or suppressing video segments to vectors. Importantly, the nodes in our matrixial graph are capable of including matrices of different shapes, and thus nodes of a matrix graph can directly represent all frame-level 2D facial feature maps (or images themselves) of an entire video regardless of its length. Then, our MGNN is the first GNN that can jointly process matrixial graphs containing varying numbers of nodes, which further learns matrix-style edge features, thereby facilitating to explicit model video-level multi-scale spatio-temporal facial behaviours among matrixial graph nodes for depression assessment. Experiments show that the explicit spatio-temporal modeling on 2D facial feature maps, facilitated by our matrixial graph/MGNN, provided significant benefits, leading our approach to achieve new state-of-the-art performances on AVEC2013 and AVEC2014 datasets with large advantages.
Leveraging Asynchronous Spiking Neural Networks for Ultra Efficient Event-Based Visual Processing
Authors: DingYi Zeng; Yuchen Wang; Honglin Cao; Wanlong Liu; Yichen Xiao; ChengzhuoLu; Wenyu Chen; Malu Zhang; Guoqing Wang; Yang Yang
Venue: AAAI 2025 (main)

Abstract: Event cameras encode visual information by generating asynchronous and sparse event streams, which hold great potential for low latency and low power consumption. Despite many successful implementations of event camera-based applications, most of them accumulate the events into frames and then utilize conventional frame-based computer vision algorithms. These frame-based methods, though typically effective, diminish the inherent advantages of the event camera's low latency and low power consumption. To solve the above problems, we propose ASGCN, which efficiently processes data on an event-by-event basis and dynamically evolves into a corresponding dynamic representation, enabling low latency and high sparsity of data representation. The sparsity computation is further improved by introducing brain-inspired spiking neural networks, resulting in low power consumption for ASGCN. Extensive and diverse experiments demonstrate the energy efficiency and low latency advantages of our processing pipeline. Especially on real-world event camera datasets, our pipeline consumes more than 10,000 times less energy and achieves similar performance compared to current frame-based methods.
Personalized Dynamic Music Emotion Recognition with Dual-Scale Attention-Based Meta-Learning
Authors: Dengming Zhang; Weitao You; Ziheng Liu; Lingyun Sun; Pei Chen
Venue: AAAI 2025 (main)

Abstract: Dynamic Music Emotion Recognition (DMER) aims to predict the emotion of different moments in music, playing a crucial role in music information retrieval. The existing DMER methods struggle to capture long-term dependencies when dealing with sequence data, which limits their performance. Furthermore, these methods often overlook the influence of individual differences on emotion perception, even though everyone has their own personalized emotional perception in the real world. Motivated by these issues, we explore more effective sequence processing methods and introduce the Personalized DMER (PDMER) problem, which requires models to predict emotions that align with personalized perception. Specifically, we propose a Dual-Scale Attention-Based Meta-Learning (DSAML) method. This method fuses features from a dual-scale feature extractor and captures both short and long-term dependencies using a dual-scale attention transformer, improving the performance in traditional DMER. To achieve PDMER, we design a novel task construction strategy that divides tasks by annotators. Samples in a task are annotated by the same annotator, ensuring consistent perception. Leveraging this strategy alongside meta-learning, DSAML can predict personalized perception of emotions with just one personalized annotation sample. Our objective and subjective experiments demonstrate that our method can achieve state-of-the-art performance in both traditional DMER and PDMER.
Learning Concept Prerequisite Relation via Global Knowledge Relation Optimization
Authors: Miao Zhang; Jiawei Wang; Kui Xiao; Shihui Wang; Yan Zhang; Hao Chen; Zhifei Li
Venue: AAAI 2025 (main)

Abstract: Learning concept prerequisite relations helps better master and build a logically coherent knowledge structure. Many studies use graph neural networks to create heterogeneous knowledge networks that enhance concept representations. However, different types of relations in these networks can influence each other. Existing research often focuses solely on concept relations, neglecting other types of knowledge connections. To address this issue, this paper proposes a novel concept prerequisite relation learning model, named the Global Knowledge Relation Optimization Model(GKROM). Specifically, we capture the impact of different knowledge relation types on document and concept semantic representations separately, integrating the document and concept semantic representations. Then, we introduce multi-objective learning to optimize the knowledge relation network from a global perspective. Through the above optimization, GKROM learns richer semantic representations for concepts and documents, improving the accuracy of concept prerequisite relation learning. Extensive experiments on public datasets demonstrate the effectiveness of our GKROM, achieving state-of-the-art performance in concept prerequisite relation learning.
SalM²: An Extremely Lightweight Saliency Mamba Model for Real-Time Cognitive Awareness of Driver Attention
Authors: Chunyu Zhao; Wentao Mu; Xian Zhou; Wenbo Liu; Fei Yan; Tao Deng
Venue: AAAI 2025 (main)

Abstract: Driver attention recognition in driving scenarios is a popular direction in traffic scene perception technology. It aims to understand human driver attention to focus on specific targets/objects in the driving scene. However, traffic scenes contain not only a large amount of visual information but also semantic information related to driving tasks. Existing methods lack attention to the actual semantic information present in driving scenes. Additionally, the traffic scene is a complex and dynamic process that requires constant attention to objects related to the current driving task. Existing models, influenced by their foundational frameworks, tend to have large parameter counts and complex structures. Therefore, this paper proposes a real-time saliency Mamba network based on the latest Mamba framework. As shown in Figure 1, our model uses very few parameters (0.08M, only 0.09~11.16% of other models), while maintaining SOTA performance or achieving over 98% of the SOTA model's performance.