OpenRaiser/Intern-Atlas huggingface dataset:
LLM-as-a-Judge in Entity Retrieval: Assessing Explicit and Implicit Relevance
Authors: Mohammad Hossein Saliminabi, Negar Arabzadeh, Seyed Mohammad Hosseini, Dimitrios Androutsos, Morteza Zihayat, Ebrahim Bagheri
Venue: CIKM
Year: 2025

Entity retrieval plays a critical role in information access systems, yet the development and evaluation of retrieval models remain constrained by the limited availability of high-quality supervision. While recent work has demonstrated the utility of large language models (LLMs) as relevance assessors in passage and document retrieval, their reliability in the context of entity retrieval-where targets are abstract, underspecified, and often semantically sparse-remains unexplored. In this work, we evaluate LLM-based judgments against two complementary supervision signals: human-annotated relevance labels from the DBpedia-Entity benchmark and implicit feedback from user clicks in the LaQuE dataset. We show that LLMs exhibit strong agreement with expert annotations and replicate user click patterns with over 91% agreement, suggesting alignment with behavioral judgments despite noisy input queries. We further identify and analyze systematic mismatches for user clicks on irrelevant entities. Our findings establish LLMs not only as effective annotators for entity relevance judgment-even when given only the entity title-but also as powerful tools for predicting click-through behavior and simulating explainable user intent. Our code, prompts, and data are publicly available at: https://github.com/17shiraz/ClickLLM
CondFairGen A Fair Conditional Generator for Tabular Data via Adaptive Sampling
Authors: David Sanchez Jr., Anantaa Kotal
Venue: CIKM
Year: 2025

Recent advances in synthetic data generation have enabled high-fidelity modeling of tabular datasets, yet fairness remains a peripheral concern, often addressed through architectural modifications or fairness-aware loss functions. We introduce CondFairGen, a fairness-aware generative model that enforces group fairness through dynamic control of conditional exposure during training. Rather than altering the model architecture or objective, CondFairGen reweights the sampling distribution over conditioning vectors based on disparity metrics across protected attributes and their intersections. This reweighting increases exposure to underrepresented or high-disparity subgroups, guiding the model toward fairer conditional distributions. By embedding fairness directly into the training schedule, CondFairGen offers a principled alternative to adversarial debiasing or post hoc correction. Empirical evaluations on standard tabular benchmarks demonstrate that CondFairGen substantially improves both marginal and intersectional fairness metrics while preserving downstream utility. These results establish conditional exposure as a practical and effective mechanism for fairness intervention in generative modeling.
HF-RAG: Hierarchical Fusion-based RAG with Multiple Sources and Rankers
Authors: Payel Santra, Madhusudan Ghosh, Debasis Ganguly, Partha Basuchowdhuri, Sudip Kumar Naskar
Venue: CIKM
Year: 2025

Leveraging both labeled (input-output associations) and unlabeled data (wider contextual grounding) may provide complementary benefits in retrieval augmented generation (RAG). However, effectively combining evidence from these heterogeneous sources is challenging as the respective similarity scores are not inter-comparable. Additionally, aggregating beliefs from the outputs of multiple rankers can improve the effectiveness of RAG. Our proposed method first aggregates the top-documents from a number of IR models using a standard rank fusion technique for each source (labeled and unlabeled). Next, we standardize the retrieval score distributions within each source by applying z-score transformation before merging the top-retrieved documents from the two sources. We evaluate our approach on the fact verification task, demonstrating that it consistently improves over the best-performing individual ranker or source and also shows better out-of-domain generalization.
Effect of Model Merging in Domain-Specific Ad-hoc Retrieval
Authors: Taiga Sasaki, Takehiro Yamamoto, Hiroaki Ohshima, Sumio Fujita
Venue: CIKM
Year: 2025

In this study, we evaluate the effect of model merging in ad-hoc retrieval tasks. Model merging is a technique that combines the diverse characteristics of multiple models. We hypothesized that applying model merging to domain-specific ad-hoc retrieval tasks could improve retrieval effectiveness. To verify this hypothesis, we merged the weights of a source retrieval model and a domain-specific (non-retrieval) model using a linear interpolation approach. A key advantage of our approach is that it requires no additional fine-tuning of the models. We conducted two experiments each in the medical and Japanese domains. The first compared the merged model with the source retrieval model, and the second compared it with a LoRA fine-tuned model under both full and limited data settings for model construction. The experimental results indicate that model merging has the potential to produce more effective domain-specific retrieval models than the source retrieval model, and may serve as a practical alternative to LoRA fine-tuning, particularly when only a limited amount of data is available.
cMALC-D: Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending
Authors: Anirudh Satheesh, Keenan Powell, Hua Wei
Venue: CIKM
Year: 2025

Many multi-agent reinforcement learning (MARL) algorithms are trained in fixed simulation environments, making them brittle when deployed in real-world scenarios with more complex and uncertain conditions. Contextual MARL (cMARL) addresses this by parameterizing environments with context variables and training a context-agnostic policy that performs well across all environment configurations. Existing cMARL methods attempt to use curriculum learning to help train and evaluate context-agnostic policies, but they often rely on unreliable proxy signals, such as value estimates or generalized advantage estimates that are noisy and unstable in multi-agent settings due to inter-agent dynamics and partial observability. To address these issues, we propose Contextual Multi-Agent LLM-Guided Curriculum Learning with Diversity-Based Context Blending (cMALC-D), a framework that uses Large Language Models (LLMs) to generate semantically meaningful curricula and provide a more robust evaluation signal. To prevent mode collapse and encourage exploration, we introduce a novel diversity-based context blending mechanism that creates new training scenarios by combining features from prior contexts. Experiments in traffic signal control domains demonstrate that cMALC-D improves both generalization and sample efficiency compared to existing curriculum learning baselines.
Membership Inference Attack Vulnerabilities of Record Linkage Models
Authors: Piyumi Seneviratne, Dinusha Vatsalan, Dali Kaafar
Venue: CIKM
Year: 2025

Record linkage plays a crucial role in integrating health, legal, and administrative data. In domains such as healthcare, unstructured records, including clinical notes, contain rich information, making their integration valuable for applications like clinical trials. While deep learning models have improved linkage quality, their privacy risks remain under-explored. We present what is, to our knowledge, the first systematic study of membership inference attack vulnerabilities in record linkage models trained on de-identified texts. Unlike traditional classifiers, linkage models operate on record pairs, prompting a rethinking of what constitutes membership leakage. Does it occur if only a record pair was seen together during training or individually? Or if neither was seen, but the pair resembles known patterns? We introduce a black-box attack based on semantic perturbation sensitivity, requiring no access to the model's internals. Our findings expose a previously unaddressed membership inference vulnerability in record linkage models: black-box attacks, even with a simple threshold-based attack model, achieved up to 92% precision and AUC 0.79. Across record linkage models trained on MIMIC-IV and PMC-Patients datasets, we observe that perturbing training-seen phrases causes significantly larger confidence shifts (e.g., Δ s = -0.648) and higher change in predicted label (label flip from 0 = non-match to 1 = match and vice versa) (e.g., 68.75%) compared to unseen variants. These preliminary results reveal membership inference vulnerabilities in text-based linkage systems, highlighting the need for deeper investigation into privacy risks, motivating new lines of privacy defences for pairwise models.
Do LLMs Dream of Electric Emotions? Towards Quantifying Metacognition and Generalizing the Teacher-Student Model Using Ensembles of LLMs
Authors: Ricky J. Sethi, Hefei Qiu, Charles Courchaine, Joshua Iacoboni
Venue: CIKM
Year: 2025

In this paper, we propose a novel framework for quantifying metacognitive processes in ensembles of Large Language Models (LLMs) and extending the traditional teacher-student model through the lens of dual-process cognitive theory. We introduce a Metacognitive State Vector (MSV) that operationalizes metacognition across five dimensions: emotional response analysis, correctness evaluation, experiential matching, conflicting information estimation, and problem importance task prioritization. In our formulation, the rapid, intuitive thinking of System 1 is mapped onto a smaller ''student'' (single LLM or ensemble of bagged LLMs) while the deliberate, analytical reasoning of System 2 is mapped onto a larger ''teacher'' (ensemble of boosted LLMs) using the MSV. Additionally, we utilize a graph-theoretic architecture to model ensemble interactions, enabling LLMs to assume dynamic roles and transition between System 1 and System 2 for improved decision-making. We view this work as a first step towards establishing a true measure of emergent metacognition in such systems.
STGS: Spatio-temporal Graph Sparsification Using Reinforcement Learning
Authors: Nasrin Shabani, Amin Beheshti, Yuankai Qi, Venus Haghighi, Jin Foo, Jia Wu
Venue: CIKM
Year: 2025

Spatio-temporal graphs encode dynamic interactions across space and time, but their size and complexity pose challenges for analysis and computation. Graph sparsification provides an effective solution to these issues by reducing the number of edges while preserving the essential structural and dynamic properties of the network. This reduction is crucial for enhancing the interpretability of complex graphs, revealing hidden patterns, and enabling more efficient computational analysis. However, real-world graphs often exhibit continuous spatial and temporal evolution, which most existing sparsification algorithms, primarily designed for static graphs, fail to address. We introduce STGS (Spatio-Temporal Graph Sparsification), a reinforcement learning-based framework for sparsifying spatio-temporal graphs. By learning to prune edges while preserving key spatio-temporal patterns, STGS enables efficient analysis of evolving systems. Experiments on real-world datasets demonstrate that STGS outperforms existing methods in both structural preservation and downstream forecasting tasks.
General Adaptive Memory Allocation for Learned Bloom Filters
Authors: You Shang, Xiang He, Ruiyuan Li, Yingying Sun, Guanyao Li, Guangchao Yang, Junbo Zhang, Yu Zheng
Venue: CIKM
Year: 2025

Membership testing, which determines whether an element belongs to a set, is widely used in fields like database systems and network applications. Bloom Filters (BFs) can solve this problem efficiently but suffer from high False Positive Rates (FPRs) and large memory requirements for massive datasets. Learned Bloom Filters (LBFs), combining a learning model with a backup Bloom Filter, mitigate these issues by capturing data distributions. However, the critical problem of memory allocation between the learning model and the backup filter has usually been overlooked, despite its significant impact on LBF performance under constrained budgets. To this end, we propose Gama, the first General Adaptive Memory Allocation framework for LBFs as far as we know. Gama introduces two memory allocation strategies: Loop-Based method and Bayesian-Based method. Loop-Based method evaluates all configurations at each training epoch, making it well-suited for scenarios with tight memory constraints. However, it faces efficiency challenges under large memory budgets due to the requirement for exhaustive evaluations. In contrast, Bayesian-Based method efficiently navigates the search space through probabilistic exploration, which reduces the number of configurations evaluated and significantly improves the efficiency while maintaining FPRs. Furthermore, we propose a hybrid approach that combines their strengths to dynamically adapt to different constraints. Experiments on three real-world datasets show that Gama can achieve a relative performance improvement of 69% in terms of FPR in the best case.
Real-E: A Foundation Benchmark for Advancing Robust and Generalizable Electricity Forecasting
Authors: Chen Shao, Michael Färber, Sebastian Pütz, Benjamin Schäfer, Yue Wang, Tobias Käfer, Zhanbo Huang, Zhenyi Zhu
Venue: CIKM
Year: 2025

Energy forecasting is vital for grid reliability and operational efficiency. Although recent advances in time series forecasting have led to progress, existing benchmarks remain limited in spatial and temporal scope and lack multi-energy features. This raises concerns about their reliability and applicability in real-world deployment. To address this, we present the Real-E dataset, covering over 74 power stations across 30+ European countries over a 10-year span with rich metadata. Using Real-E, we conduct an extensive data analysis and benchmark over 20 baselines across various model types. We introduce a new metric to quantify shifts in correlation structures and show that existing methods struggle on our dataset, which exhibits more complex and non-stationary correlation dynamics. Our findings highlight key limitations of current methods and offer a strong empirical basis for building more robust forecasting models.
The Metadata Impedance Mismatch between Databases and Programming Languages
Authors: Vishal Sharma, Curtis E. Dyreson
Venue: CIKM
Year: 2025

This paper identifies a problem with databases that support metadata. Previous research has proposed annotating values stored in a database with metadata, such as time, security, privacy, and quality. The metadata influences how a value is used. For example, sequenced temporal semantics proscribes comparing a value to one alive at a different time. But when values stored in a database are pulled into the realm of a programming language from a database through an API, web service, or in a user-defined function, a step-down transformation of the data occurs. The transformation strips the metadata, changing the semantics of the value. The metadata is discarded because a programming language process a scalar value, not one annotated with metadata. This metadata-related impedance mismatch between databases and programming languages limits the real-world adoption of databases that support metadata.
FAIR Data Assessment Using LLMs: The Fair-Way
Authors: Anmol Sharma, Sulayman K. Sowe, Soo-Yon Kim, Sayed Hoseini, Fidan Limani, Zeyd Boukhers, Christoph Lange, Stefan Decker
Venue: CIKM
Year: 2025

As part of modern research practices, the FAIR data principles have become essential for data discoverability, usability, and sharing. Existing implementations for automatically assessing FAIR adherence (FAIRness) often suffer from limited usability, inconsistent accuracy, and difficult-to-interpret results, as they require explicit rules to cover for specific FAIR assessment frameworks, which are not easy to generalize. This paper introduces Fair-Way, an open source tool that leverages Large Language Models (LLMs) to automate FAIRness assessment. Fair-Way applies a divide-and-conquer approach to decompose the assessment process into fine-grained tasks, as well as to split the metadata into manageable chunks. Evaluation demonstrates that Fair-Way achieves performance comparable to existing tools, while outperforming them in several key metrics. Moreover, Fair-Way generalizes across FAIR assessment indicators without requiring explicitly programmed logic and supports both structured and unstructured metadata in diverse formats. Finally, it enables user-defined, domain-specific tests, which are typically not supported by other systems. Overall, Fair-Way represents a scalable and flexible solution to accelerate FAIR data practices across research domains.
Towards Few-shot Chemical Reaction Outcome Prediction
Authors: Yili Shen, Yijun Tian, Cheng-Wei Ju, Olaf Wiest, Xiangliang Zhang
Venue: CIKM
Year: 2025

Accurate chemical reaction prediction is essential for drug discovery and synthetic planning. However, this task becomes particularly challenging in low-data scenarios, where novel reaction types lack sufficient training examples. To address this challenge, we propose FewRxn, a novel model-agnostic few-shot reaction prediction framework that enables rapid adaptation to unseen reaction types using only a few training samples. FewRxn integrates several key innovations, including segmentation masks for enhanced reactant representation, fingerprint embeddings for richer molecular context, and task-aware meta-learning for effective knowledge transfer. Through extensive evaluations, FewRxn achieves state-of-the-art accuracy in few-shot settings, significantly outperforming traditional fine-tuning methods. Additionally, our work provides insights into the impact of molecular representations on reaction knowledge transfer, demonstrating that knowledge captured under molecular graph-based formulation consistently outperforms those learned in forms of SMILES generation in few-shot learning.
Retrieval-Augmented Image Captioning via Synthesized Entity-Aware Knowledge Representations
Authors: Lin Shen, Chenxu Cui, Jinchao Zhang, Haihui Fan, Haotian Jin, Bo Li
Venue: CIKM
Year: 2025

Retrieval-Augmented Image Captioning enhances the model's understanding of real-world images by retrieving external knowledge. Existing methods mainly use original captions or isolated entities related to the query image to help generate captions. However, these methods make the model either imitate the caption style or fail to capture the relationship between entities, resulting in a lack of diversity or inaccuracy in the generated captions. To address these issues, we propose SEAR, a novel framework that utilizes external Synthesized Entity-Aware knowledge Representations to improve captioning performance. Specifically, SEAR clusters images based on scene-level and entity-level features, and synthesizes each clustered images into representative images as retrieval indexes, and simultaneously utilizes a large model to extract and supplement structured knowledge graphs from the corresponding cluster captions. Furthermore, we design a knowledge-graph pruner to prune the knowledge graph by retaining the most relevant subgraphs to the query image. By undertaking these steps in an integrated manner, SEAR enables the model to acquire non-redundant and structured information for generating captions and avoid data-related privacy issues. Extensive experiments on MSCOCO, Flickr30k, and NoCaps demonstrate the effectiveness of our method both in-domain and out-of-domain, outperforming existing lightweight RAIC methods and remaining competitive with heavyweight models.
EmoPerso: Enhancing Personality Detection with Self-Supervised Emotion-Aware Modelling
Authors: Lingzhi Shen, Xiaohao Cai, Yunfei Long, Imran Razzak, Guanming Chen, Shoaib Jameel
Venue: CIKM
Year: 2025

Personality detection from text is commonly performed by analysing users' social media posts. However, existing methods heavily rely on large-scale annotated datasets, making it challenging to obtain high-quality personality labels. Moreover, most studies treat emotion and personality as independent variables, overlooking their interactions. In this paper, we propose a novel self-supervised framework, EmoPerso, which improves personality detection through emotion-aware modelling. EmoPerso first leverages generative mechanisms for synthetic data augmentation and rich representation learning. It then extracts pseudo-labeled emotion features and jointly optimizes them with personality prediction via multi-task learning. A cross-attention module is employed to capture fine-grained interactions between personality traits and the inferred emotional representations. To further refine relational reasoning, EmoPerso adopts a self-taught strategy to enhance the model's reasoning capabilities iteratively. Extensive experiments on two benchmark datasets demonstrate that EmoPerso surpasses state-of-the-art models. The source code is available at https://github.com/slz0925/EmoPerso.
ROKAN: Toward Interpretable and Domain-Robust Memory Behavior Modeling
Authors: Xiaoxuan Shen, Zhihai Hu, Di Chen, Jianwen Sun, Shengyingjie Liu
Venue: CIKM
Year: 2025

Memory behavior modeling aims to predict individual performance over time and uncover underlying cognitive mechanisms. However, existing approaches often struggle to balance predictive accuracy, domain generalization, and model interpretability. To address this, we propose ROKAN, a cognitively inspired and symbolically interpretable memory modeling framework. Based on the Multiscale Context Model, ROKAN formalizes the evolution of memory traces as a differentiable Ordinary Differential Equation system, implemented via Kolmogorov-Arnold Networks to derive human-readable symbolic expressions. To enhance generalization across heterogeneous learning domains, we design an Adaptive Domain-Aware loss function, which integrates Empirical Risk Minimization with Distributionally Robust Optimization through dynamic domain-aware weighting. Our experiments demonstrate that ROKAN significantly outperforms existing mainstream methods in both predictive accuracy and domain generalization. The symbolic expressions were found to exhibit formal consistency with classical memory theories, which lends support to the model's theoretical assumptions and empirical performance, and provides a new pathway toward theoretically grounded white-box memory modeling. Our code is available at https://github.com/hellowads/ROKAN.
Bridging the Gap Between Sparsity and Redundancy: A Dual-Decoding Framework with Global Context for Map Inference
Authors: Yudong Shen, Jiali Mao, Wenyu Wu, Yixiao Tong, Guoping Liu, Chaoya Wang
Venue: CIKM
Year: 2025

Trajectory data has become a key resource for automated map inference due to its low cost, broad coverage, and continuous availability. However, uneven trajectory density often leads to fragmented roads in sparse areas and redundant segments in dense regions, posing significant challenges for existing methods. To address these issues, we propose DGMap, a dual-decoding framework with global context awareness, featuring Multi-scale Grid Encoding, Mask-enhanced Keypoint Extraction, and Global Context-aware Relation Prediction. By integrating global semantic context with local geometric features, DGMap improves keypoint detection accuracy to reduce road fragmentation in sparse-trajectory areas. Additionally, the Global Context-aware Relation Prediction module suppresses false connections in dense-trajectory regions by modeling long-range trajectory patterns.Experimental results on three real-world datasets show that DGMap outperforms state-of-the-art methods by 5% in APLS, with notable performance gains on trajectory data from the Didi Chuxing platform.