pwc-archive/papers-with-abstracts huggingface dataset:
Generative Retrieval Meets Multi-Graded Relevance
Authors: Yubao Tang, Ruqing Zhang, Jiafeng Guo, Maarten de Rijke, Wei Chen, Xueqi Cheng
Date: 2024-09-27 00:00:00
Tasks: Decoder, Information Retrieval, Retrieval

Generative retrieval represents a novel approach to information retrieval. It uses an encoder-decoder architecture to directly produce relevant document identifiers (docids) for queries. While this method offers benefits, current approaches are limited to scenarios with binary relevance data, overlooking the potential for documents to have multi-graded relevance. Extending generative retrieval to accommodate multi-graded relevance poses challenges, including the need to reconcile likelihood probabilities for docid pairs and the possibility of multiple relevant documents sharing the same identifier. To address these challenges, we introduce a framework called GRaded Generative Retrieval (GR$^2$). GR$^2$ focuses on two key components: ensuring relevant and distinct identifiers, and implementing multi-graded constrained contrastive training. First, we create identifiers that are both semantically relevant and sufficiently distinct to represent individual documents effectively. This is achieved by jointly optimizing the relevance and distinctness of docids through a combination of docid generation and autoencoder models. Second, we incorporate information about the relationship between relevance grades to guide the training process. We use a constrained contrastive training strategy to bring the representations of queries and the identifiers of their relevant documents closer together, based on their respective relevance grades. Extensive experiments on datasets with both multi-graded and binary relevance demonstrate the effectiveness of GR$^2$.
Evaluation of Cluster Id Assignment Schemes with ABCDE
Authors: Stephan van Staden
Date: 2024-09-26 00:00:00
Tasks: Clustering

A cluster id assignment scheme labels each cluster of a clustering with a distinct id. The goal of id assignment is semantic id stability, which means that, whenever possible, a cluster for the same underlying concept as that of a historical cluster should ideally receive the same id as the historical cluster. Semantic id stability allows the users of a clustering to refer to a concept's cluster with an id that is stable across clusterings/time. This paper treats the problem of evaluating the relative merits of id assignment schemes. In particular, it considers a historical clustering with id assignments, and a new clustering with ids assigned by a baseline and an experiment. It produces metrics that characterize both the magnitude and the quality of the id assignment diffs between the baseline and the experiment. That happens by transforming the problem of cluster id assignment into a problem of cluster membership, and evaluating it with ABCDE. ABCDE is a sophisticated and scalable technique for evaluating differences in cluster membership in real-world applications, where billions of items are grouped into millions of clusters, and some items are more important than others. The paper also describes several generalizations to the basic evaluation setup for id assignment schemes. For example, it is fairly straightforward to evaluate changes that simultaneously mutate cluster memberships and cluster ids. The ideas are generously illustrated with examples.
Mitigating LLM Hallucinations via Conformal Abstention
Authors: Yasin Abbasi Yadkori, Ilja Kuzborskij, David Stutz, András György, Adam Fisch, Arnaud Doucet, Iuliya Beloshapka, Wei-Hung Weng, Yao-Yuan Yang, Csaba Szepesvári, Ali Taylan Cemgil, Nenad Tomasev
Date: 2024-04-04 00:00:00
Tasks: Conformal Prediction, Generative Question Answering, Hallucination, Language Modelling, Large Language Model, Question Answering, Temporal Sequences, TriviaQA

We develop a principled procedure for determining when a large language model (LLM) should abstain from responding (e.g., by saying "I don't know") in a general domain, instead of resorting to possibly "hallucinating" a non-sensical or incorrect answer. Building on earlier approaches that use self-consistency as a more reliable measure of model confidence, we propose using the LLM itself to self-evaluate the similarity between each of its sampled responses for a given query. We then further leverage conformal prediction techniques to develop an abstention procedure that benefits from rigorous theoretical guarantees on the hallucination rate (error rate). Experimentally, our resulting conformal abstention method reliably bounds the hallucination rate on various closed-book, open-domain generative question answering datasets, while also maintaining a significantly less conservative abstention rate on a dataset with long responses (Temporal Sequences) compared to baselines using log-probability scores to quantify uncertainty, while achieveing comparable performance on a dataset with short answers (TriviaQA). To evaluate the experiments automatically, one needs to determine if two responses are equivalent given a question. Following standard practice, we use a thresholded similarity function to determine if two responses match, but also provide a method for calibrating the threshold based on conformal prediction, with theoretical guarantees on the accuracy of the match prediction, which might be of independent interest.
Forecasting Macroeconomic Dynamics using a Calibrated Data-Driven Agent-based Model
Authors: Samuel Wiese, Jagoda Kaszowska-Mojsa, Joel Dyer, Jose Moran, Marco Pangallo, Francois Lafond, John Muellbauer, Anisoara Calinescu, J. Doyne Farmer
Date: 2024-09-27 00:00:00
Tasks: Bayesian Inference, Time Series, Time Series Forecasting

In the last few years, economic agent-based models have made the transition from qualitative models calibrated to match stylised facts to quantitative models for time series forecasting, and in some cases, their predictions have performed as well or better than those of standard models (see, e.g. Poledna et al. (2023a); Hommes et al. (2022); Pichler et al. (2022)). Here, we build on the model of Poledna et al., adding several new features such as housing markets, realistic synthetic populations of individuals with income, wealth and consumption heterogeneity, enhanced behavioural rules and market mechanisms, and an enhanced credit market. We calibrate our model for all 38 OECD member countries using state-of-the-art approximate Bayesian inference methods and test it by making out-of-sample forecasts. It outperforms both the Poledna and AR(1) time series models by a highly statistically significant margin. Our model is built within a platform we have developed, making it easy to build, run, and evaluate alternative models, which we hope will encourage future work in this area.
Can AI Enhance its Creativity to Beat Humans ?
Authors: Anne-Gaëlle Maltese Pierre Pelletier, Rémy Guichardaz
Date: 2024-09-27 00:00:00

Creativity is a fundamental pillar of human expression and a driving force behind innovation, yet it now stands at a crossroads. As artificial intelligence advances at an astonishing pace, the question arises: can machines match and potentially surpass human creativity? This study investigates the creative performance of artificial intelligence (AI) compared to humans by analyzing the effects of two distinct prompting strategies (a Naive and an Expert AI) on AI and across three different tasks (Text, Draw and Alternative Uses tasks). Human external evaluators have scored creative outputs generated by humans and AI, and these subjective creative scores were complemented with objective measures based on quantitative measurements and NLP tools. The results reveal that AI generally outperforms humans in creative tasks, though this advantage is nuanced by the specific nature of each task and the chosen creativity criteria. Ultimately, while AI demonstrates superior performance in certain creative domains, our results suggest that integrating human feedback is crucial for maximizing AI's creative potential.
Cognitive phantoms in LLMs through the lens of latent variables
Authors: Sanne Peereboom, Inga Schwabe, Bennett Kleinberg
Date: 2024-09-06 00:00:00

Large language models (LLMs) increasingly reach real-world applications, necessitating a better understanding of their behaviour. Their size and complexity complicate traditional assessment methods, causing the emergence of alternative approaches inspired by the field of psychology. Recent studies administering psychometric questionnaires to LLMs report human-like traits in LLMs, potentially influencing LLM behaviour. However, this approach suffers from a validity problem: it presupposes that these traits exist in LLMs and that they are measurable with tools designed for humans. Typical procedures rarely acknowledge the validity problem in LLMs, comparing and interpreting average LLM scores. This study investigates this problem by comparing latent structures of personality between humans and three LLMs using two validated personality questionnaires. Findings suggest that questionnaires designed for humans do not validly measure similar constructs in LLMs, and that these constructs may not exist in LLMs at all, highlighting the need for psychometric analyses of LLM responses to avoid chasing cognitive phantoms. Keywords: large language models, psychometrics, machine behaviour, latent variable modeling, validity
Asymptotic tracking control of dynamic reference over homomorphically encrypted data with finite modulus
Authors: Shuai Feng, Junsoo Kim
Date: 2024-09-27 00:00:00
Tasks: Quantization

This paper considers a tracking control problem, in which the dynamic controller is encrypted with an additively homomorphic encryption scheme and the output of a process tracks a dynamic reference asymptotically. Our paper is motivated by the following problem: When dealing with both asymptotic tracking and dynamic reference, we find that the control input is generally subject to overflow issues under a finite modulus, though the dynamic controller consists of only integer coefficients. First, we provide a new controller design method such that the coefficients of the tracking controller can be transformed into integers leveraging the zooming-in factor of dynamic quantization. By the Cayley-Hamilton theorem, we represent the control input as linear combination of the previous control inputs. Leveraging the property above, we design an algorithm on the actuator side such that it can restore the control input from the lower bits under a finite modulus. A lower bound of the modulus is also provided. As an extension of the first result, we further solve the problem of unbounded internal state taking place in the actuator. In particular, the actuator can restore the correct control input under the same modulus. A simulation example is provided to verify the control schemes proposed in our paper.
Statistical analysis on the effectiveness of a low insulin index, alkaline and functional diet evaluated with machine learning techniques
Authors: Fabiana Antoniali, Maria Luisa Conza, Francesco Alessandro Conventi, Francesco Cirotto, Antonio D'Avanzo, Agostino De Iorio, Annalisa De Iorio, Nunzia Formicola, Manuela Forte, Federica Miele, Giovanni Perna, Biagio Rossi, Elvira Rossi
Date: 2024-09-26 00:00:00

In this paper, a statistical analysis of the performance of a low insulin index, alkaline and functional diet developed by ANTUR evaluated with machine learning techniques is reported. The sample of patients was checked on a regular basis with a BioImpedenziometric (BIA) analysis. The BIA gives about 40 parameters output describing the health status of the patient. The sample of 1626 patients was grouped in clusters with similar characteristics by a neural network algorithm. A study of the behaviour of the BIA parameters evolution over time with respect to the first visit was performed. More than the 75% of the patient that followed the ANTUR diet showed an improvement of the health status.
ECG Signal Preprocessing and SVM Classifier-Based Abnormality Detection in Remote Healthcare Applications
Authors: C. Venkatesan, P. Karthigaikumar, Anand Paul, S. Satheeskumaran, R. Kumar
Date: 2018-01-22 00:00:00
Proceeding: journal 2018 1
Tasks: Anomaly Detection, Classification

Medical expert systems are part of the portable and smart healthcare monitoring devices used in day-to-day life. Arrhythmic beat classification is mainly used in electrocardiogram (ECG) abnormality detection for identifying heart related problems. In this paper, ECG signal preprocessing and support vector machine-based arrhythmic beat classification are performed to categorize into normal and abnormal subjects. In ECG signal preprocessing, a delayed error normalized LMS adaptive filter is used to achieve high speed and low latency design with less computational elements. Since the signal processing technique is developed for remote healthcare systems, white noise removal is mainly focused. Discrete wavelet transform is applied on the preprocessed signal for HRV feature extraction and machine learning techniques are used for performing arrhythmic beat classification. In this paper, SVM classifier and other popular classifiers have been used on noise removed feature extracted signal for beat classification. Results indicate that the performance of SVM classifier is better than other machine learning-based classifiers.
Machine Learning-Based Cardiovascular Disease Detection Using Optimal Feature Selection
Authors: Tahseen Ullah, Syed Irfan Ullah, Khalil Ullah, Muhammad Ishaq, Ahmad Khan
Date: 2024-01-13 00:00:00
Proceeding: journal 2024 1
Tasks: feature selection

Cardiovascular disease (CVD) is a prevalent and serious condition causing a significant global mortality rate. According to the World Health Organization (WHO), in 2022, CVD claimed the lives of approximately 19.1 million people, accounting for 33% of global fatalities. ECG is widely used for automatic detection of CVD using traditional machine learning; however, it is usually difficult to select optimal features. Addressing this issue, a scalable machine learning-based architecture is proposed for early CVD detection based optimal feature selection. This architecture aims to revolutionize healthcare by enabling timely diagnosis and treatment, reducing CVD-related fatalities. Comprising data collection, storage, and processing components, the system employs machine learning classifiers to predict patients’ heart conditions. Initially features are extracted from ECG signals then feature selection techniques like FCBF, MrMr, and relief, along with PSO-optimization are used to select optimal features. Extra Tree and Random Forest classifiers trained on the selected features have achieved notable performance rates with accuracy of 100% respectively. Furthermore, a comparison of the proposed method with state of the art on both small and large dataset is provided. The proposed system holds potential to revolutionize patient care and substantially lower CVD-related mortality, enhancing the quality of life for affected individuals. In summary, this architecture offers a promising solution to the pressing issue of CVD and paves the way for advanced healthcare systems.
Towards Automated Patent Workflows: AI-Orchestrated Multi-Agent Framework for Intellectual Property Management and Analysis
Authors: Sakhinana Sagar Srinivas, Vijay Sri Vaikunth, Venkataramana Runkana
Date: 2024-09-21 00:00:00
Tasks: Abstractive Text Summarization, Knowledge Graphs, Management, Patent classification, Retrieval-augmented Generation, Semantic Similarity, Semantic Textual Similarity

Patents are the currency of innovation, and like any currency, they need to be managed and protected (Gavin Potenza). Patents, as legal documents that secure intellectual property rights, play a critical role in technological innovation. The growing complexity of patent documents and the surge in patent applications have created a need for automated solutions in patent analysis. In this work, we present PatExpert, an autonomous multi-agent conversational framework designed to streamline and optimize patent-related tasks. The framework consists of a metaagent that coordinates task-specific expert agents for various patent-related tasks and a critique agent for error handling and feedback provision. The meta-agent orchestrates specialized expert agents, each fine-tuned for specific tasks such as patent classification, acceptance, claim generation, abstractive summarization, multi-patent analysis, and scientific hypothesis generation. For multi-patent analysis, the framework incorporates advanced methods like Graph Retrieval-Augmented Generation (GRAG) to enhance response accuracy and relevance by combining semantic similarity with knowledge graphs. Error handling is managed by critique agents (Gold-LLM-as-a-Judge and Reward-LLM-as-a-Judge), which evaluate output responses for accuracy and provide iterative feedback. The framework also prioritizes explainability, ensuring transparent justifications for decisions made during patent analysis. Its comprehensive capabilities make it a valuable tool for automating complex patent workflows, enhancing efficiency, accuracy, and compliance in patent-related tasks. Empirical evidence demonstrates significant improvements in patent processing tasks, concluding that the framework offers a robust solution for automating and optimizing patent analysis.
Utilizing Priors in Sampling-based Cost Minimization
Authors: Yuan-Yao Lou, Jonathan Spencer, Kwang Taik Kim, Mung Chiang
Date: 2024-09-30 00:00:00

We consider an autonomous vehicle (AV) agent performing a long-term cost-minimization problem in the elapsed time $T$ over sequences of states $s_{1:T}$ and actions $a_{1:T}$ for some fixed, known (though potentially learned) cost function $C(s_t,a_t)$, approximate system dynamics $P$, and distribution over initial states $d_0$. The goal is to minimize the expected cost-to-go of the driving trajectory $\tau = s_1, a_1, ..., s_T, a_T$ from the initial state.
All-in-One Image Coding for Joint Human-Machine Vision with Multi-Path Aggregation
Authors: Xu Zhang, Peiyao Guo, Ming Lu, Zhan Ma
Date: 2024-09-29 00:00:00
Tasks: All, Data Compression, Feature Importance, Image Classification, Image Compression, Semantic Segmentation

Image coding for multi-task applications, catering to both human perception and machine vision, has been extensively investigated. Existing methods often rely on multiple task-specific encoder-decoder pairs, leading to high overhead of parameter and bitrate usage, or face challenges in multi-objective optimization under a unified representation, failing to achieve both performance and efficiency. To this end, we propose Multi-Path Aggregation (MPA) integrated into existing coding models for joint human-machine vision, unifying the feature representation with an all-in-one architecture. MPA employs a predictor to allocate latent features among task-specific paths based on feature importance varied across tasks, maximizing the utility of shared features while preserving task-specific features for subsequent refinement. Leveraging feature correlations, we develop a two-stage optimization strategy to alleviate multi-task performance degradation. Upon the reuse of shared features, as low as 1.89% parameters are further augmented and fine-tuned for a specific task, which completely avoids extensive optimization of the entire model. Experimental results show that MPA achieves performance comparable to state-of-the-art methods in both task-specific and multi-objective optimization across human viewing and machine analysis tasks. Moreover, our all-in-one design supports seamless transitions between human- and machine-oriented reconstruction, enabling task-controllable interpretation without altering the unified model. Code is available at https://github.com/NJUVISION/MPA.
PALM: Few-Shot Prompt Learning for Audio Language Models
Authors: Asif Hanif, Maha Tufail Agro, Mohammad Areeb Qazi, Hanan Aldarmaki
Date: 2024-09-29 00:00:00
Tasks: Few-Shot Learning, Prompt Learning

Audio-Language Models (ALMs) have recently achieved remarkable success in zero-shot audio recognition tasks, which match features of audio waveforms with class-specific text prompt features, inspired by advancements in Vision-Language Models (VLMs). Given the sensitivity of zero-shot performance to the choice of hand-crafted text prompts, many prompt learning techniques have been developed for VLMs. We explore the efficacy of these approaches in ALMs and propose a novel method, Prompt Learning in Audio Language Models (PALM), which optimizes the feature space of the text encoder branch. Unlike existing methods that work in the input space, our approach results in greater training efficiency. We demonstrate the effectiveness of our approach on 11 audio recognition datasets, encompassing a variety of speech-processing tasks, and compare the results with three baselines in a few-shot learning setup. Our method is either on par with or outperforms other approaches while being computationally less demanding. Code is available at https://asif-hanif.github.io/palm/
Towards Open-Vocabulary Semantic Segmentation Without Semantic Labels
Authors: Heeseong Shin, Chaehyun Kim, Sunghwan Hong, Seokju Cho, Anurag Arnab, Paul Hongsuck Seo, Seungryong Kim
Date: 2024-09-30 00:00:00
Tasks: Online Clustering, Open Vocabulary Semantic Segmentation, Open-Vocabulary Semantic Segmentation, Semantic Segmentation

Large-scale vision-language models like CLIP have demonstrated impressive open-vocabulary capabilities for image-level tasks, excelling in recognizing what objects are present. However, they struggle with pixel-level recognition tasks like semantic segmentation, which additionally require understanding where the objects are located. In this work, we propose a novel method, PixelCLIP, to adapt the CLIP image encoder for pixel-level understanding by guiding the model on where, which is achieved using unlabeled images and masks generated from vision foundation models such as SAM and DINO. To address the challenges of leveraging masks without semantic labels, we devise an online clustering algorithm using learnable class names to acquire general semantic concepts. PixelCLIP shows significant performance improvements over CLIP and competitive results compared to caption-supervised methods in open-vocabulary semantic segmentation. Project page is available at https://cvlab-kaist.github.io/PixelCLIP
Brain-JEPA: Brain Dynamics Foundation Model with Gradient Positioning and Spatiotemporal Masking
Authors: Zijian Dong, Ruilin Li, Yilei Wu, Thuan Tinh Nguyen, Joanna Su Xian Chong, Fang Ji, Nathanael Ren Jie Tong, Christopher Li Hsian Chen, Juan Helen Zhou
Date: 2024-09-28 00:00:00
Tasks: Prognosis

We introduce Brain-JEPA, a brain dynamics foundation model with the Joint-Embedding Predictive Architecture (JEPA). This pioneering model achieves state-of-the-art performance in demographic prediction, disease diagnosis/prognosis, and trait prediction through fine-tuning. Furthermore, it excels in off-the-shelf evaluations (e.g., linear probing) and demonstrates superior generalizability across different ethnic groups, surpassing the previous large model for brain activity significantly. Brain-JEPA incorporates two innovative techniques: Brain Gradient Positioning and Spatiotemporal Masking. Brain Gradient Positioning introduces a functional coordinate system for brain functional parcellation, enhancing the positional encoding of different Regions of Interest (ROIs). Spatiotemporal Masking, tailored to the unique characteristics of fMRI data, addresses the challenge of heterogeneous time-series patches. These methodologies enhance model performance and advance our understanding of the neural circuits underlying cognition. Overall, Brain-JEPA is paving the way to address pivotal questions of building brain functional coordinate system and masking brain activity at the AI-neuroscience interface, and setting a potentially new paradigm in brain activity analysis through downstream adaptation.