pwc-archive/papers-with-abstracts huggingface dataset:
A Counterfactual Explanation Framework for Retrieval Models
Authors: Bhavik Chandna, Procheta Sen
Date: 2024-09-01 00:00:00
Tasks: counterfactual, Counterfactual Explanation, Information Retrieval, Retrieval

Explainability has become a crucial concern in today's world, aiming to enhance transparency in machine learning and deep learning models. Information retrieval is no exception to this trend. In existing literature on explainability of information retrieval, the emphasis has predominantly been on illustrating the concept of relevance concerning a retrieval model. The questions addressed include why a document is relevant to a query, why one document exhibits higher relevance than another, or why a specific set of documents is deemed relevant for a query. However, limited attention has been given to understanding why a particular document is not favored (e.g. not within top-K) with respect to a query and a retrieval model. In an effort to address this gap, our work focus on the question of what terms need to be added within a document to improve its ranking. This in turn answers the question of which words played a role in not being favored in the document by a retrieval model for a particular query. We use a counterfactual framework to solve the above-mentioned research problem. To the best of our knowledge, we mark the first attempt to tackle this specific counterfactual problem (i.e. examining the absence of which words can affect the ranking of a document). Our experiments show the effectiveness of our proposed approach in predicting counterfactuals for both statistical (e.g. BM25) and deep-learning-based models (e.g. DRMM, DSSM, ColBERT, MonoT5). The code implementation of our proposed approach is available in https://anonymous.4open.science/r/CfIR-v2.
Fair Reciprocal Recommendation in Matching Markets
Authors: Yoji Tomita, Tomohiki Yokoyama
Date: 2024-09-01 00:00:00
Tasks: Fairness, Recommendation Systems

Recommender systems play an increasingly crucial role in shaping people's opportunities, particularly in online dating platforms. It is essential from the user's perspective to increase the probability of matching with a suitable partner while ensuring an appropriate level of fairness in the matching opportunities. We investigate reciprocal recommendation in two-sided matching markets between agents divided into two sides. In our model, a match is considered successful only when both individuals express interest in each other. Additionally, we assume that agents prefer to appear prominently in the recommendation lists presented to those on the other side. We define each agent's opportunity to be recommended and introduce its fairness criterion, envy-freeness, from the perspective of fair division theory. The recommendations that approximately maximize the expected number of matches, empirically obtained by heuristic algorithms, are likely to result in significant unfairness of opportunity. Therefore, there can be a trade-off between maximizing the expected matches and ensuring fairness of opportunity. To address this challenge, we propose a method to find a policy that is close to being envy-free by leveraging the Nash social welfare function. Experiments on synthetic and real-world datasets demonstrate the effectiveness of our approach in achieving both relatively high expected matches and fairness for opportunities of both sides in reciprocal recommender systems.
A Learnable Agent Collaboration Network Framework for Personalized Multimodal AI Search Engine
Authors: Yunxiao Shi, Min Xu, Haimin Zhang, Xing Zi, Qiang Wu
Date: 2024-09-01 00:00:00
Tasks: AI Agent, RAG, Retrieval-augmented Generation

Large language models (LLMs) and retrieval-augmented generation (RAG) techniques have revolutionized traditional information access, enabling AI agent to search and summarize information on behalf of users during dynamic dialogues. Despite their potential, current AI search engines exhibit considerable room for improvement in several critical areas. These areas include the support for multimodal information, the delivery of personalized responses, the capability to logically answer complex questions, and the facilitation of more flexible interactions. This paper proposes a novel AI Search Engine framework called the Agent Collaboration Network (ACN). The ACN framework consists of multiple specialized agents working collaboratively, each with distinct roles such as Account Manager, Solution Strategist, Information Manager, and Content Creator. This framework integrates mechanisms for picture content understanding, user profile tracking, and online evolution, enhancing the AI search engine's response quality, personalization, and interactivity. A highlight of the ACN is the introduction of a Reflective Forward Optimization method (RFO), which supports the online synergistic adjustment among agents. This feature endows the ACN with online learning capabilities, ensuring that the system has strong interactive flexibility and can promptly adapt to user feedback. This learning method may also serve as an optimization approach for agent-based systems, potentially influencing other domains of agent applications.
Genetic Approach to Mitigate Hallucination in Generative IR
Authors: Hrishikesh Kulkarni, Nazli Goharian, Ophir Frieder, Sean MacAvaney
Date: 2024-08-25 00:00:00
Tasks: Answer Generation, Hallucination

Generative language models hallucinate. That is, at times, they generate factually flawed responses. These inaccuracies are particularly insidious because the responses are fluent and well-articulated. We focus on the task of Grounded Answer Generation (part of Generative IR), which aims to produce direct answers to a user's question based on results retrieved from a search engine. We address hallucination by adapting an existing genetic generation approach with a new 'balanced fitness function' consisting of a cross-encoder model for relevance and an n-gram overlap metric to promote grounding. Our balanced fitness function approach quadruples the grounded answer generation accuracy while maintaining high relevance.
Evolving Text Data Stream Mining
Authors: Jay Kumar
Date: 2024-08-15 00:00:00
Tasks: Multi-Label Learning

A text stream is an ordered sequence of text documents generated over time. A massive amount of such text data is generated by online social platforms every day. Designing an algorithm for such text streams to extract useful information is a challenging task due to unique properties of the stream such as infinite length, data sparsity, and evolution. Thereby, learning useful information from such streaming data under the constraint of limited time and memory has gained increasing attention. During the past decade, although many text stream mining algorithms have proposed, there still exists some potential issues. First, high-dimensional text data heavily degrades the learning performance until the model either works on subspace or reduces the global feature space. The second issue is to extract semantic text representation of documents and capture evolving topics over time. Moreover, the problem of label scarcity exists, whereas existing approaches work on the full availability of labeled data. To deal with these issues, in this thesis, new learning models are proposed for clustering and multi-label learning on text streams.
Web Retrieval Agents for Evidence-Based Misinformation Detection
Authors: Jacob-Junqi Tian, Hao Yu, Yury Orlovskiy, Tyler Vergho, Mauricio Rivera, Mayank Goel, Zachary Yang, Jean-Francois Godbout, Reihaneh Rabbany, Kellin Pelrine
Date: 2024-08-15 00:00:00
Tasks: Fact Checking, Misinformation, Retrieval

This paper develops an agent-based automated fact-checking approach for detecting misinformation. We demonstrate that combining a powerful LLM agent, which does not have access to the internet for searches, with an online web search agent yields better results than when each tool is used independently. Our approach is robust across multiple models, outperforming alternatives and increasing the macro F1 of misinformation detection by as much as 20 percent compared to LLMs without search. We also conduct extensive analyses on the sources our system leverages and their biases, decisions in the construction of the system like the search tool and the knowledge base, the type of evidence needed and its impact on the results, and other parts of the overall process. By combining strong performance with in-depth understanding, we hope to provide building blocks for future search-enabled misinformation mitigation systems.
A load forecasting approach for integrated energy systems based on aggregation hybrid modal decomposition and combined model
Authors: Haoyu Chen, Hai Huang, Yong Zheng, Bing Yang
Date: 2024-08-12 00:00:00
Proceeding: Applied Energy 2024 8
Tasks: Load Forecasting

To improve the granularity of load decomposition within integrated energy system (IES), a novel multiple load forecasting approach is proposed. This method leverages an aggregated hybrid modal decomposition (AHMD) strategy and a composite model approach to thoroughly investigate the inherent characteristics of load profiles. Improved Complete Ensemble Empirical Mode Decomposition with Adaptive Noise (ICEEMDAN) is utilized for the primary decomposition of electric, cooling, and heating loads, yielding a series of subsequences. To further refine the analysis, Fuzzy Dispersion Entropy (FDE), and Over-zero Rate (OZR) are employed to aggregate the decomposed subsequences. Subsequently, Successive Variational Mode Decomposition (SVMD) is applied to break down the high-frequency components in a secondary manner. The adoption of the rolling decomposition strategy effectively circumvents the information leakage problem caused by traditional modal decomposition-prediction methods. In concert with environmental features, the Multiple Linear Regression (MLR) model forecasts the low-frequency components. Meanwhile, the Temporal Convolutional Network-Bidirectional Gated Recurrent Unit (TCN-BiGRU) fusion neural network is deployed for predicting mid-frequency and high-frequency components. Additionally, a Multi-Head Attention (MHA) mechanism is introduced to effectively assign weights to the features. The predictions of each component are superimposed to obtain the final forecast. An example of an integrated campus energy system is used to compare the predictive effectiveness of different models. The results show that the proposed method exhibits superior prediction accuracy across multiple load types. This approach significantly enhances energy efficiency and fosters the advancement of sustainable energy practices.
Short-term power load forecasting for integrated energy system based on a residual and attentive LSTM-TCN hybrid network
Authors: Hongyi Li, Shenhao Li, Yuxin Wu, Yue Xiao, Zhichong Pan, Min Liu
Date: 2024-05-10 00:00:00
Proceeding: Energy Res 2024 5
Tasks: Demand Forecasting, energy management, Load Forecasting

In the context of Integrated Energy System (IES), accurate short-term power demand forecasting is crucial for ensuring system reliability, optimizing operational efficiency through resource allocation, and supporting effective real-time decision-making in energy management. However, achieving high forecasting accuracy faces significant challenges due to the inherent complexity and stochastic nature of IES’s short-term load profiles, resulting from diverse consumption patterns among end-users and the intricate coupling within the network of interconnected energy sources. To address this issue, a dedicated Short-Term Power Load Forecasting (STPLF) framework for IES is proposed, which relies on a newly developed hybrid deep learning architecture. The framework seamlessly combines Long Short-Term Memory (LSTM) with Temporal Convolutional Network (TCN), enhanced by an attention mechanism module. By merging these methodologies, the network leverages the parallel processing prowess of TCN alongside LSTM’s ability to retain long-range temporal information, thus enabling it to dynamically concentrate on relevant sections of time series data. This synergy leads to improved prediction accuracy and broader applicability. Furthermore, the integration of residual connections within the network structure serves to deepen its learning capabilities and enhance overall performance. Ultimately, results from a real case study of a user-level IES demonstrate that the Mean Absolute Percentage Error (MAPE) of the proposed framework on the test set is 2.35%. This error rate is lower than the averages of traditional methods (3.43%) and uncombined single submodules (2.80%).
Zero-Shot Visual Reasoning by Vision-Language Models: Benchmarking and Analysis
Authors: Aishik Nagar, Shantanu Jaiswal, Cheston Tan
Date: 2024-08-27 00:00:00
Tasks: Benchmarking, Large Language Model, Question Answering, Visual Question Answering, Visual Question Answering (VQA), Visual Reasoning, World Knowledge

Vision-language models (VLMs) have shown impressive zero- and few-shot performance on real-world visual question answering (VQA) benchmarks, alluding to their capabilities as visual reasoning engines. However, the benchmarks being used conflate "pure" visual reasoning with world knowledge, and also have questions that involve a limited number of reasoning steps. Thus, it remains unclear whether a VLM's apparent visual reasoning performance is due to its world knowledge, or due to actual visual reasoning capabilities. To clarify this ambiguity, we systematically benchmark and dissect the zero-shot visual reasoning capabilities of VLMs through synthetic datasets that require minimal world knowledge, and allow for analysis over a broad range of reasoning steps. We focus on two novel aspects of zero-shot visual reasoning: i) evaluating the impact of conveying scene information as either visual embeddings or purely textual scene descriptions to the underlying large language model (LLM) of the VLM, and ii) comparing the effectiveness of chain-of-thought prompting to standard prompting for zero-shot visual reasoning. We find that the underlying LLMs, when provided textual scene descriptions, consistently perform better compared to being provided visual embeddings. In particular, 18% higher accuracy is achieved on the PTR dataset. We also find that CoT prompting performs marginally better than standard prompting only for the comparatively large GPT-3.5-Turbo (175B) model, and does worse for smaller-scale models. This suggests the emergence of CoT abilities for visual reasoning in LLMs at larger scales even when world knowledge is limited. Overall, we find limitations in the abilities of VLMs and LLMs for more complex visual reasoning, and highlight the important role that LLMs can play in visual reasoning.
Trainable pruned ternary quantization for medical signal classification models
Authors: Yamil Vindas, Blaise Kévin Guépié, Marilys Almar, Emmanuel Roux, Philippe Delachartre
Date: 2024-10-01 00:00:00
Proceeding: Neurocomputing 2024 10
Tasks: Model Compression, Quantization

The field of deep learning is renowned for its resource-intensive nature, hence improving its environmental impact is crucial. In this paper, we propose a novel model compression method to mitigate the energy demands of deep learning for a greener, and more sustainable AI landscape. Our approach relies on an asymmetric weakly-differentiable pruning function that leverages weight statistics to directly incorporate adaptable pruning into the quantization mechanism. This enables us to achieve higher compression rates globally while simultaneously reducing energy consumption and minimizing classification performance degradation. The efficacy of our approach was evaluated using three distinct models on three distinct datasets: cerebral emboli (HITS), epileptic seizure recognition (ESR), and MNIST. Our method demonstrated a superior balance between compression, energy consumption, and classification performance compared to other state-of-the-art extreme quantization methods, across all models and datasets. In fact, on the HITS dataset with a two-dimensional convolutional neural network, we achieved strong gains of 50.6%, 54.9%, 52.1% in compression rates (of the global model and the quantized layers only, respectively) and energy consumption, respectively, while improving the Matthews correlation coefficient by 2.5% compared to other approaches. The code is available at: https://github.com/yamilvindas/pTTQ.
Semi-automatic data annotation based on feature-space projection and local quality metrics: An application to cerebral emboli characterization
Authors: Yamil Vindas, Blaise Kévin Guépié, Marilys Almar, Emmanuel Roux, Philippe Delachartre
Date: 2022-07-01 00:00:00
Proceeding: Medical Image Analysis 2022 7

We propose a semi-supervised learning approach to annotate a dataset with reduced requirements for manual annotation and with controlled annotation error. The method is based on feature-space projection and label propagation using local quality metrics. First, an auto-encoder extracts the features of the samples in an unsupervised manner. Then, the extracted features are projected by a t-distributed stochastic neighbor embedding algorithm into a two-dimensional (2D) space. A selection of the best 2D projection is introduced based on the silhouette score. The expert annotator uses the obtained 2D representation to manually label samples. Finally, the labels of the labeled samples are propagated to the unlabeled samples using a K-nearest neighbor strategy and local quality metrics. We compare our method against semi-supervised optimum-path forest and K-nearest neighbor label propagation (without considering local quality metrics). Our method achieves state-of-the-art results on three different datasets by labeling more than of the samples with an annotation error from to . Additionally, our method allows to control the trade-off between annotation error and number of labeled samples. Moreover, we combine our method with robust loss functions to compensate for the label noise introduced by automatic label propagation. Our method allows to achieve similar, and even better, classification performances compared to those obtained using a fully manually labeled dataset, with up to in terms of classification accuracy.
Guided deep embedded clustering regularization for multifeature medical signal classification
Authors: Yamil Vindas, Emmanuel Roux, Blaise Kévin Guépié, Marilys Almar, Philippe Delachartre
Date: 2023-11-01 00:00:00
Proceeding: Pattern Recognition 2023 11
Tasks: Classification, Clustering

Medical signal classification often focuses on one representation (raw signal or time frequency). In that context, recent works have shown the value of exploiting different representations simultaneously. We propose a regularized end-to-end trained model for classification in a medical context exploiting both the raw signal and a time-frequency representation (TFR). First, a 2D convolutional neural network (CNN) encoder and a 1D CNN-transformer encoder start by extracting embedded representations from the TFR and the raw signal, respectively. Then, the obtained embeddings are fused to form a common latent space that is used for classification. We propose to guide the training of each encoder by applying two iterated losses. Moreover, we propose to regularize the fused common latent space using deep embedded clustering. Extensive experiments on three medical datasets and ablation studies show the adaptability and good performance of our method for medical signal classification. Our method makes it possible to improve the classification performance from to MCC on a transcranial Doppler dataset, when compared with single-feature counterparts, while giving more stable models. The code is available at: https://github.com/gdec-submission/gdec/.
Deep Embedded Clustering Regularization for Supervised Imbalanced Cerebral Emboli Classification Using Transcranial Doppler Ultrasound
Authors: Yamil Vindas, Emmanuel Roux, Blaise Kévin Guépié, Marilys Almar, Philippe Delachartre
Date: 2023-11-01 00:00:00
Proceeding: European Signal Processing Conference 2023 11

High intensity transient signals (HITS) can be detected using transcranial Doppler ultrasound monitoring to help stroke prevention. The various types of HITS are by nature imbalanced: solid emboli are rare events, compared to artifacts or gaseous emboli. Therefore, when training deep models on these data, one have to take into account the class imbalance. In this work, we propose a deep embedded clustering (DEC) based regularization technique, DEC-R to handle imbalanced datasets. Our proposed method decompose a classification model into an encoder and a classifier, with DEC-R applied to the former during supervised training. We validate our method on four synthetic datasets and one HITS datasets. The results show that our approach is robust against imbalanced datasets, allowing to increase the classification performances of different models on both, imbalanced (maximum imbalance ratio of 20) and balanced datasets. We achieve an increase of 4.86% and 1.27% in terms of Matthews correlation coefficient on two imbalanced datasets, a synthetic dataset (of 3D points) and a HITS clinical dataset, respectively. Our code can be found in https://github.com/yamilvindas/imbalanced_dec_regularization.
Global–Local Information Soft-Alignment for Cross-Modal Remote-Sensing Image–Text Retrieval
Authors: Gang Hu, Zaidao Wen, Yafei Lv, Jianting Zhang, Qian Wu
Date: 2024-05-14 00:00:00
Proceeding: journal 2024 5
Tasks: Cross-Modal Retrieval, Cross-Modal Retrieval on RSITMD, Image-text Retrieval, Retrieval, Text Retrieval

Cross-modal remote-sensing image–text retrieval (CMRSITR) is a challenging task that aims to retrieve target remote-sensing (RS) images based on textual descriptions. However, the modal gap between texts and RS images poses a significant challenge. RS images comprise multiple targets and complex backgrounds, necessitating the mining of both global and local information (GaLR) for effective CMRSITR. Existing approaches primarily focus on local image features while disregarding the local features of the text and their correspondence. These methods typically fuse global and local image features and align them with global text features. However, they struggle to eliminate the influence of cluttered backgrounds and may overlook crucial targets. To address these limitations, we propose a novel framework for CMRSITR based on a transformer architecture, which leverages global–local information soft alignment (GLISA) to enhance retrieval performance. Our framework incorporates a global image extraction module, which captures the global semantic features of image–text pairs and effectively represents the relationships among multiple targets in RS images. In addition, we introduce an adaptive local information extraction (ALIE) module that adaptively mines discriminative local clues from both RS images and texts, aligning the corresponding fine-grained information. To mitigate semantic ambiguities during the alignment of local features, we design a local information soft-alignment (LISA) module. In comparative evaluations using two public CMRSITR datasets, our proposed method achieves state-of-the-art results, surpassing not only traditional cross-modal retrieval methods by a substantial margin but also other contrastive language-image pretraining (CLIP)-based methods.