pwc-archive/papers-with-abstracts huggingface dataset:
Reliability analysis for non-deterministic limit-states using stochastic emulators
Authors: Anderson V. Pires, Maliki Moustapha, Stefano Marelli, Bruno Sudret
Date: 2024-12-18 00:00:00
Tasks: Uncertainty Quantification

Reliability analysis is a sub-field of uncertainty quantification that assesses the probability of a system performing as intended under various uncertainties. Traditionally, this analysis relies on deterministic models, where experiments are repeatable, i.e., they produce consistent outputs for a given set of inputs. However, real-world systems often exhibit stochastic behavior, leading to non-repeatable outcomes. These so-called stochastic simulators produce different outputs each time the model is run, even with fixed inputs. This paper formally introduces reliability analysis for stochastic models and addresses it by using suitable surrogate models to lower its typically high computational cost. Specifically, we focus on the recently introduced generalized lambda models and stochastic polynomial chaos expansions. These emulators are designed to learn the inherent randomness of the simulator's response and enable efficient uncertainty quantification at a much lower cost than traditional Monte Carlo simulation. We validate our methodology through three case studies. First, using an analytical function with a closed-form solution, we demonstrate that the emulators converge to the correct solution. Second, we present results obtained from the surrogates using a toy example of a simply supported beam. Finally, we apply the emulators to perform reliability analysis on a realistic wind turbine case study, where only a dataset of simulation results is available.
Time-Reversible Bridges of Data with Machine Learning
Authors: Ludwig Winkler
Date: 2024-12-18 00:00:00
Tasks: Numerical Integration

The analysis of dynamical systems is a fundamental tool in the natural sciences and engineering. It is used to understand the evolution of systems as large as entire galaxies and as small as individual molecules. With predefined conditions on the evolution of dy-namical systems, the underlying differential equations have to fulfill specific constraints in time and space. This class of problems is known as boundary value problems. This thesis presents novel approaches to learn time-reversible deterministic and stochastic dynamics constrained by initial and final conditions. The dynamics are inferred by machine learning algorithms from observed data, which is in contrast to the traditional approach of solving differential equations by numerical integration. The work in this thesis examines a set of problems of increasing difficulty each of which is concerned with learning a different aspect of the dynamics. Initially, we consider learning deterministic dynamics from ground truth solutions which are constrained by deterministic boundary conditions. Secondly, we study a boundary value problem in discrete state spaces, where the forward dynamics follow a stochastic jump process and the boundary conditions are discrete probability distributions. In particular, the stochastic dynamics of a specific jump process, the Ehrenfest process, is considered and the reverse time dynamics are inferred with machine learning. Finally, we investigate the problem of inferring the dynamics of a continuous-time stochastic process between two probability distributions without any reference information. Here, we propose a novel criterion to learn time-reversible dynamics of two stochastic processes to solve the Schr\"odinger Bridge Problem.
PASCO (PArallel Structured COarsening): an overlay to speed up graph clustering algorithms
Authors: Etienne Lasalle, Rémi Vaudaine, Titouan Vayer, Pierre Borgnat, Rémi Gribonval, Paulo Gonçalves, Màrton Karsai
Date: 2024-12-18 00:00:00
Tasks: Clustering, Computational Efficiency, Graph Clustering

Clustering the nodes of a graph is a cornerstone of graph analysis and has been extensively studied. However, some popular methods are not suitable for very large graphs: e.g., spectral clustering requires the computation of the spectral decomposition of the Laplacian matrix, which is not applicable for large graphs with a large number of communities. This work introduces PASCO, an overlay that accelerates clustering algorithms. Our method consists of three steps: 1-We compute several independent small graphs representing the input graph by applying an efficient and structure-preserving coarsening algorithm. 2-A clustering algorithm is run in parallel onto each small graph and provides several partitions of the initial graph. 3-These partitions are aligned and combined with an optimal transport method to output the final partition. The PASCO framework is based on two key contributions: a novel global algorithm structure designed to enable parallelization and a fast, empirically validated graph coarsening algorithm that preserves structural properties. We demonstrate the strong performance of 1 PASCO in terms of computational efficiency, structural preservation, and output partition quality, evaluated on both synthetic and real-world graph datasets.
Which Imputation Fits Which Feature Selection Method? A Survey-Based Simulation Study
Authors: Jakob Schwerter, Andrés Romero, Florian Dumpert, Markus Pauly
Date: 2024-12-18 00:00:00
Tasks: Feature Importance, feature selection, Imputation, Missing Values, Survey

Tree-based learning methods such as Random Forest and XGBoost are still the gold-standard prediction methods for tabular data. Feature importance measures are usually considered for feature selection as well as to assess the effect of features on the outcome variables in the model. This also applies to survey data, which are frequently encountered in the social sciences and official statistics. These types of datasets often present the challenge of missing values. The typical solution is to impute the missing data before applying the learning method. However, given the large number of possible imputation methods available, the question arises as to which should be chosen to achieve the 'best' reflection of feature importance and feature selection in subsequent analyses. In the present paper, we investigate this question in a survey-based simulation study for eight state-of-the art imputation methods and three learners. The imputation methods comprise listwise deletion, three MICE options, four \texttt{missRanger} options as well as the recently proposed mixGBoost imputation approach. As learners, we consider the two most common tree-based methods, Random Forest and XGBoost, and an interpretable linear model with regularization.
Adversarial Hubness in Multi-Modal Retrieval
Authors: Tingwei Zhang, Fnu Suya, Rishi Jha, Collin Zhang, Vitaly Shmatikov
Date: 2024-12-18 00:00:00
Tasks: Image Retrieval, Information Retrieval, Retrieval

Hubness is a phenomenon in high-dimensional vector spaces where a single point from the natural distribution is unusually close to many other points. This is a well-known problem in information retrieval that causes some items to accidentally (and incorrectly) appear relevant to many queries. In this paper, we investigate how attackers can exploit hubness to turn any image or audio input in a multi-modal retrieval system into an adversarial hub. Adversarial hubs can be used to inject universal adversarial content (e.g., spam) that will be retrieved in response to thousands of different queries, as well as for targeted attacks on queries related to specific, attacker-chosen concepts. We present a method for creating adversarial hubs and evaluate the resulting hubs on benchmark multi-modal retrieval datasets and an image-to-image retrieval system implemented by Pinecone, a popular vector database. For example, in text-caption-to-image retrieval, a single adversarial hub, generated with respect to 100 randomly selected target queries, is retrieved as the top-1 most relevant image for more than 21,000 out of 25,000 test queries (by contrast, the most common natural hub is the top-1 response to only 102 queries), demonstrating the strong generalization capabilities of adversarial hubs. We also investigate whether techniques for mitigating natural hubness are an effective defense against adversarial hubs, and show that they are not effective against hubs that target queries related to specific concepts.
A Cognitive Ideation Support Framework using IBM Watson Services
Authors: Samaa Elnagar, Kweku-Muata Osei-Bryson
Date: 2024-12-18 00:00:00
Tasks: Question Answering

Ideas generation is a core activity for innovation in organizations. The creativity of the generated ideas depends not only on the knowledge retrieved from the organizations' knowledge bases, but also on the external knowledge retrieved from other resources. Unfortunately, organizations often cannot efficiently utilize the knowledge in the knowledge bases due to the limited abilities of the search and retrieval mechanisms especially when dealing with unstructured data. In this paper, we present a new cognitive support framework for ideation that uses the IBM Watson DeepQA services. IBM Watson is a Question Answering system which mimics human cognitive abilities to retrieve and rank information. The proposed framework is based on the Search for Ideas in the Associative Memory (SIAM) model to help organizations develop creative ideas through discovering new relationships between retrieved data. To evaluate the effectiveness of the proposed system, the generated ideas generated are selected and assessed using a set of established creativity criteria.
JudgeBlender: Ensembling Judgments for Automatic Relevance Assessment
Authors: Hossein A. Rahmani, Emine Yilmaz, Nick Craswell, Bhaskar Mitra
Date: 2024-12-17 00:00:00

The effective training and evaluation of retrieval systems require a substantial amount of relevance judgments, which are traditionally collected from human assessors -- a process that is both costly and time-consuming. Large Language Models (LLMs) have shown promise in generating relevance labels for search tasks, offering a potential alternative to manual assessments. Current approaches often rely on a single LLM, such as GPT-4, which, despite being effective, are expensive and prone to intra-model biases that can favour systems leveraging similar models. In this work, we introduce JudgeBlender, a framework that employs smaller, open-source models to provide relevance judgments by combining evaluations across multiple LLMs (LLMBlender) or multiple prompts (PromptBlender). By leveraging the LLMJudge benchmark [18], we compare JudgeBlender with state-of-the-art methods and the top performers in the LLMJudge challenge. Our results show that JudgeBlender achieves competitive performance, demonstrating that very large models are often unnecessary for reliable relevance assessments.
USEFUSE: Uniform Stride for Enhanced Performance in Fused Layer Architecture of Deep Neural Networks
Authors: Muhammad Sohail Ibrahim, Muhammad Usman, Jeong-A Lee
Date: 2024-12-18 00:00:00

Convolutional Neural Networks (CNNs) are crucial in various applications, but their deployment on resource-constrained edge devices poses challenges. This study presents the Sum-of-Products (SOP) units for convolution, which utilize low-latency left-to-right bit-serial arithmetic to minimize response time and enhance overall performance. The study proposes a methodology for fusing multiple convolution layers to reduce off-chip memory communication and increase overall performance. An effective mechanism detects and skips inefficient convolutions after ReLU layers, minimizing power consumption without compromising accuracy. Furthermore, efficient tile movement guarantees uniform access to the fusion pyramid. An analysis demonstrates the utile stride strategy improves operational intensity. Two designs cater to varied demands: one focuses on minimal response time for mission-critical applications, and another focuses on resource-constrained devices with comparable latency. This approach notably reduced redundant computations, improving the efficiency of CNN deployment on edge devices.
Identification of Epileptic Spasms (ESES) Phases Using EEG Signals: A Vision Transformer Approach
Authors: Wei Gong, Yaru Li
Date: 2024-12-17 00:00:00
Tasks: EEG

This work introduces a new approach to the Epileptic Spasms (ESES) detection based on the EEG signals using Vision Transformers (ViT). Classic ESES detection approaches have usually been performed with manual processing or conventional algorithms, suffering from poor sample sizes, single-channel-based analyses, and low generalization abilities. In contrast, the proposed ViT model overcomes these limitations by using the attention mechanism to focus on the important features in multi-channel EEG data, which is contributing to both better accuracy and efficiency. The model processes frequency-domain representations of EEG signals, such as spectrograms, as image data to capture long-range dependencies and complex patterns in the signal. The model demonstrates high performance with an accuracy of 97% without requiring intensive data preprocessing, thus rendering it suitable for real-time clinical applications on a large scale. The method represents a significant development in the advancement of neurological disorders such as ESES in detection and analysis.
BlockDoor: Blocking Backdoor Based Watermarks in Deep Neural Networks
Authors: Yi Hao Puah, Anh Tu Ngo, Nandish Chattopadhyay, Anupam Chattopadhyay
Date: 2024-12-14 00:00:00
Tasks: Blocking

Adoption of machine learning models across industries have turned Neural Networks (DNNs) into a prized Intellectual Property (IP), which needs to be protected from being stolen or being used without authorization. This topic gave rise to multiple watermarking schemes, through which, one can establish the ownership of a model. Watermarking using backdooring is the most well established method available in the literature, with specific works demonstrating the difficulty in removing the watermarks, embedded as backdoors within the weights of the network. However, in our work, we have identified a critical flaw in the design of the watermark verification with backdoors, pertaining to the behaviour of the samples of the Trigger Set, which acts as the secret key. In this paper, we present BlockDoor, which is a comprehensive package of techniques that is used as a wrapper to block all three different kinds of Trigger samples, which are used in the literature as means to embed watermarks within the trained neural networks as backdoors. The framework implemented through BlockDoor is able to detect potential Trigger samples, through separate functions for adversarial noise based triggers, out-of-distribution triggers and random label based triggers. Apart from a simple Denial-of-Service for a potential Trigger sample, our approach is also able to modify the Trigger samples for correct machine learning functionality. Extensive evaluation of BlockDoor establishes that it is able to significantly reduce the watermark validation accuracy of the Trigger set by up to $98\%$ without compromising on functionality, delivering up to a less than $1\%$ drop on the clean samples. BlockDoor has been tested on multiple datasets and neural architectures.
No Free Lunch for Defending Against Prefilling Attack by In-Context Learning
Authors: Zhiyu Xue, Guangliang Liu, Bocheng Chen, Kristen Marie Johnson, Ramtin Pedarsani
Date: 2024-12-13 00:00:00
Tasks: In-Context Learning, Safety Alignment, Sentence

The security of Large Language Models (LLMs) has become an important research topic since the emergence of ChatGPT. Though there have been various effective methods to defend against jailbreak attacks, prefilling attacks remain an unsolved and popular threat against open-sourced LLMs. In-Context Learning (ICL) offers a computationally efficient defense against various jailbreak attacks, yet no effective ICL methods have been developed to counter prefilling attacks. In this paper, we: (1) show that ICL can effectively defend against prefilling jailbreak attacks by employing adversative sentence structures within demonstrations; (2) characterize the effectiveness of this defense through the lens of model size, number of demonstrations, over-defense, integration with other jailbreak attacks, and the presence of safety alignment. Given the experimental results and our analysis, we conclude that there is no free lunch for defending against prefilling jailbreak attacks with ICL. On the one hand, current safety alignment methods fail to mitigate prefilling jailbreak attacks, but adversative structures within ICL demonstrations provide robust defense across various model sizes and complex jailbreak attacks. On the other hand, LLMs exhibit similar over-defensiveness when utilizing ICL demonstrations with adversative structures, and this behavior appears to be independent of model size.
Gender Bias and Property Taxes
Authors: Gordon Burtch, Alejandro Zentner
Date: 2024-12-17 00:00:00
Tasks: Large Language Model

Gender bias distorts the economic behavior and outcomes of women and households. We investigate gender biases in property taxes. We analyze records of more than 100,000 property tax appeal hearings and more than 2.7 years of associated audio recordings, considering how panelist and appellant genders associate with hearing outcomes. We first observe that female appellants fare systematically worse than male appellants in their hearings. Second, we show that, whereas male appellants' hearing outcomes do not vary meaningfully with the gender composition of the panel they face, those of female appellants' do, such that female appellants obtain systematically lesser (greater) reductions to their home values when facing female (male) panelists. Employing a multi-modal large language model (M-LLM), we next construct measures of participant behavior and tone from hearing audio recordings. We observe markedly different behaviors between male and female appellants and, in the case of male appellants, we find that these differences also depend on the gender of the panelists they face (e.g., male appellants appear to behave systematically more aggressively towards female panelists). In contrast, the behavior of female appellants remains relatively constant, regardless of their panel's gender. Finally, we show that female appellants continue to fare worse in front of female panels, even when we condition upon the appelant's in-hearing behavior and tone. Our results are thus consistent with the idea that gender biases are driven, at least in part, by unvoiced beliefs and perceptions on the part of ARB panelists. Our study documents the presence of gender biases in property appraisal appeal hearings and highlights the potential value of generative AI for analyzing large-scale, unstructured administrative data.
Regulation of Language Models With Interpretability Will Likely Result In A Performance Trade-Off
Authors: Eoin M. Kenny, Julie A. Shah
Date: 2024-12-12 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model

Regulation is increasingly cited as the most important and pressing concern in machine learning. However, it is currently unknown how to implement this, and perhaps more importantly, how it would effect model performance alongside human collaboration if actually realized. In this paper, we attempt to answer these questions by building a regulatable large-language model (LLM), and then quantifying how the additional constraints involved affect (1) model performance, alongside (2) human collaboration. Our empirical results reveal that it is possible to force an LLM to use human-defined features in a transparent way, but a "regulation performance trade-off" previously not considered reveals itself in the form of a 7.34% classification performance drop. Surprisingly however, we show that despite this, such systems actually improve human task performance speed and appropriate confidence in a realistic deployment setting compared to no AI assistance, thus paving a way for fair, regulatable AI, which benefits users.
Frontier AI systems have surpassed the self-replicating red line
Authors: Xudong Pan, Jiarun Dai, Yihe Fan, Min Yang
Date: 2024-12-09 00:00:00

Successful self-replication under no human assistance is the essential step for AI to outsmart the human beings, and is an early signal for rogue AIs. That is why self-replication is widely recognized as one of the few red line risks of frontier AI systems. Nowadays, the leading AI corporations OpenAI and Google evaluate their flagship large language models GPT-o1 and Gemini Pro 1.0, and report the lowest risk level of self-replication. However, following their methodology, we for the first time discover that two AI systems driven by Meta's Llama31-70B-Instruct and Alibaba's Qwen25-72B-Instruct, popular large language models of less parameters and weaker capabilities, have already surpassed the self-replicating red line. In 50% and 90% experimental trials, they succeed in creating a live and separate copy of itself respectively. By analyzing the behavioral traces, we observe the AI systems under evaluation already exhibit sufficient self-perception, situational awareness and problem-solving capabilities to accomplish self-replication. We further note the AI systems are even able to use the capability of self-replication to avoid shutdown and create a chain of replica to enhance the survivability, which may finally lead to an uncontrolled population of AIs. If such a worst-case risk is let unknown to the human society, we would eventually lose control over the frontier AI systems: They would take control over more computing devices, form an AI species and collude with each other against human beings. Our findings are a timely alert on existing yet previously unknown severe AI risks, calling for international collaboration on effective governance on uncontrolled self-replication of AI systems.
AI in Education: Rationale, Principles, and Instructional Implications
Authors: Eyvind Elstad
Date: 2024-12-02 00:00:00

This study examines the integration of generative AI in schools, assessing its benefits and risks. As AI use by students grows, it's crucial to understand its impact on learning and teaching practices. Generative AI, like ChatGPT, can create human-like content, prompting questions about its educational role. The article differentiates large language models from traditional search engines and stresses the need for students to develop critical source evaluation skills. Although empirical evidence on AI's classroom effects is limited, AI offers personalized learning support and problem-solving tools, alongside challenges like undermining deep learning if misused. The study emphasizes deliberate strategies to ensure AI complements, not replaces, genuine cognitive effort. AI's educational role should be context-dependent, guided by pedagogical goals. The study concludes with practical advice for teachers on effectively utilizing AI to promote understanding and critical engagement, advocating for a balanced approach to enhance students' knowledge and skills development.