pwc-archive/papers-with-abstracts huggingface dataset:
Latent Concept-based Explanation of NLP Models
Authors: Xuemin Yu, Fahim Dalvi, Nadir Durrani, Marzia Nouri, Hassan Sajjad
Date: 2024-04-18 00:00:00

Interpreting and understanding the predictions made by deep learning models poses a formidable challenge due to their inherently opaque nature. Many previous efforts aimed at explaining these predictions rely on input features, specifically, the words within NLP models. However, such explanations are often less informative due to the discrete nature of these words and their lack of contextual verbosity. To address this limitation, we introduce the Latent Concept Attribution method (LACOAT), which generates explanations for predictions based on latent concepts. Our foundational intuition is that a word can exhibit multiple facets, contingent upon the context in which it is used. Therefore, given a word in context, the latent space derived from our training process reflects a specific facet of that word. LACOAT functions by mapping the representations of salient input words into the training latent space, allowing it to provide latent context-based explanations of the prediction.
Grammatical Error Correction for Code-Switched Sentences by Learners of English
Authors: Kelvin Wey Han Chan, Christopher Bryant, Li Nguyen, Andrew Caines, Zheng Yuan
Date: 2024-04-18 00:00:00
Tasks: Grammatical Error Correction

Code-switching (CSW) is a common phenomenon among multilingual speakers where multiple languages are used in a single discourse or utterance. Mixed language utterances may still contain grammatical errors however, yet most existing Grammar Error Correction (GEC) systems have been trained on monolingual data and not developed with CSW in mind. In this work, we conduct the first exploration into the use of GEC systems on CSW text. Through this exploration, we propose a novel method of generating synthetic CSW GEC datasets by translating different spans of text within existing GEC corpora. We then investigate different methods of selecting these spans based on CSW ratio, switch-point factor and linguistic constraints, and identify how they affect the performance of GEC systems on CSW text. Our best model achieves an average increase of 1.57 $F_{0.5}$ across 3 CSW test sets (English-Chinese, English-Korean and English-Japanese) without affecting the model's performance on a monolingual dataset. We furthermore discovered that models trained on one CSW language generalise relatively well to other typologically similar CSW languages.
NormAd: A Framework for Measuring the Cultural Adaptability of Large Language Models
Authors: Abhinav Rao, Akhila Yerukola, Vishwa Shah, Katharina Reinecke, Maarten Sap
Date: 2024-04-18 00:00:00
Tasks: Navigate, Specificity

To be effectively and safely deployed to global user populations, large language models (LLMs) must adapt outputs to user values and culture, not just know about them. We introduce NormAd, an evaluation framework to assess LLMs' cultural adaptability, specifically measuring their ability to judge social acceptability across different levels of cultural norm specificity, from abstract values to explicit social norms. As an instantiation of our framework, we create NormAd-Eti, a benchmark of 2.6k situational descriptions representing social-etiquette related cultural norms from 75 countries. Through comprehensive experiments on NormAd-Eti, we find that LLMs struggle to accurately judge social acceptability across these varying degrees of cultural contexts and show stronger adaptability to English-centric cultures over those from the Global South. Even in the simplest setting where the relevant social norms are provided, our best models' performance (<82%) lags behind humans (>95%). In settings with abstract values and country information, model performance drops substantially (<60%), while human accuracy remains high (>90%). Furthermore, we find that models are better at recognizing socially acceptable versus unacceptable situations. Our findings showcase the current pitfalls in socio-cultural reasoning of LLMs which hinder their adaptability for global audiences.
Characterizing LLM Abstention Behavior in Science QA with Context Perturbations
Authors: Bingbing Wen, Bill Howe, Lucy Lu Wang
Date: 2024-04-18 00:00:00

The correct model response in the face of uncertainty is to abstain from answering a question so as not to mislead the user. In this work, we study the ability of LLMs to abstain from answering context-dependent science questions when provided insufficient or incorrect context. We probe model sensitivity in several settings: removing gold context, replacing gold context with irrelevant context, and providing additional context beyond what is given. In experiments on four QA datasets with six LLMs, we show that performance varies greatly across models, across the type of context provided, and also by question type; in particular, many LLMs seem unable to abstain from answering boolean questions using standard QA prompts. Our analysis also highlights the unexpected impact of abstention performance on QA task accuracy. Counter-intuitively, in some settings, replacing gold context with irrelevant context or adding irrelevant context to gold context can improve abstention performance in a way that results in improvements in task performance. Our results imply that changes are needed in QA dataset design and evaluation to more effectively assess the correctness and downstream impacts of model abstention.
AmbigDocs: Reasoning across Documents on Different Entities under the Same Name
Authors: Yoonsang Lee, Xi Ye, Eunsol Choi
Date: 2024-04-18 00:00:00

Different entities with the same name can be difficult to distinguish. Handling confusing entity mentions is a crucial skill for language models (LMs). For example, given the question "Where was Michael Jordan educated?" and a set of documents discussing different people named Michael Jordan, can LMs distinguish entity mentions to generate a cohesive answer to the question? To test this ability, we introduce a new benchmark, AmbigDocs. By leveraging Wikipedia's disambiguation pages, we identify a set of documents, belonging to different entities who share an ambiguous name. From these documents, we generate questions containing an ambiguous name and their corresponding sets of answers. Our analysis reveals that current state-of-the-art models often yield ambiguous answers or incorrectly merge information belonging to different entities. We establish an ontology categorizing four types of incomplete answers and automatic evaluation metrics to identify such categories. We lay the foundation for future work on reasoning across multiple documents with ambiguous entities.
Leveraging Symbolic Regression for Heuristic Design in the Traveling Thief Problem
Authors: Andrew Ni, Lee Spector
Date: 2024-04-19 00:00:00
Tasks: regression, Symbolic Regression

The Traveling Thief Problem is an NP-hard combination of the well known traveling salesman and knapsack packing problems. In this paper, we use symbolic regression to learn useful features of near-optimal packing plans, which we then use to design efficient metaheuristic genetic algorithms for the traveling thief algorithm. By using symbolic regression again to initialize the metaheuristic GA with near-optimal individuals, we are able to design a fast, interpretable, and effective packing initialization scheme. Comparisons against previous initialization schemes validates our algorithm design.
Near-Tight Runtime Guarantees for Many-Objective Evolutionary Algorithms
Authors: Simon Wietheger, Benjamin Doerr
Date: 2024-04-19 00:00:00
Tasks: Evolutionary Algorithms

Despite significant progress in the field of mathematical runtime analysis of multi-objective evolutionary algorithms (MOEAs), the performance of MOEAs on discrete many-objective problems is little understood. In particular, the few existing bounds for the SEMO, global SEMO, and SMS-EMOA algorithms on classic benchmarks are all roughly quadratic in the size of the Pareto front. In this work, we prove near-tight runtime guarantees for these three algorithms on the four most common benchmark problems OneMinMax, CountingOnesCountingZeros, LeadingOnesTrailingZeros, and OneJumpZeroJump, and this for arbitrary numbers of objectives. Our bounds depend only linearly on the Pareto front size, showing that these MOEAs on these benchmarks cope much better with many objectives than what previous works suggested. Our bounds are tight apart from small polynomial factors in the number of objectives and length of bitstrings. This is the first time that such tight bounds are proven for many-objective uses of these MOEAs. While it is known that such results cannot hold for the NSGA-II, we do show that our bounds, via a recent structural result, transfer to the NSGA-III algorithm.
Sup3r: A Semi-Supervised Algorithm for increasing Sparsity, Stability, and Separability in Hierarchy Of Time-Surfaces architectures
Authors: Marco Rasetto, Himanshu Akolkar, Ryad Benosman
Date: 2024-04-15 00:00:00
Tasks: Incremental Learning

The Hierarchy Of Time-Surfaces (HOTS) algorithm, a neuromorphic approach for feature extraction from event data, presents promising capabilities but faces challenges in accuracy and compatibility with neuromorphic hardware. In this paper, we introduce Sup3r, a Semi-Supervised algorithm aimed at addressing these challenges. Sup3r enhances sparsity, stability, and separability in the HOTS networks. It enables end-to-end online training of HOTS networks replacing external classifiers, by leveraging semi-supervised learning. Sup3r learns class-informative patterns, mitigates confounding features, and reduces the number of processed events. Moreover, Sup3r facilitates continual and incremental learning, allowing adaptation to data distribution shifts and learning new tasks without forgetting. Preliminary results on N-MNIST demonstrate that Sup3r achieves comparable accuracy to similarly sized Artificial Neural Networks trained with back-propagation. This work showcases the potential of Sup3r to advance the capabilities of HOTS networks, offering a promising avenue for neuromorphic algorithms in real-world applications.
Items or Relations -- what do Artificial Neural Networks learn?
Authors: Renate Krause, Stefan Reimann
Date: 2024-04-15 00:00:00

What has an Artificial Neural Network (ANN) learned after being successfully trained to solve a task - the set of training items or the relations between them? This question is difficult to answer for modern applied ANNs because of their enormous size and complexity. Therefore, here we consider a low-dimensional network and a simple task, i.e., the network has to reproduce a set of training items identically. We construct the family of solutions analytically and use standard learning algorithms to obtain numerical solutions. These numerical solutions differ depending on the optimization algorithm and the weight initialization and are shown to be particular members of the family of analytical solutions. In this simple setting, we observe that the general structure of the network weights represents the training set's symmetry group, i.e., the relations between training items. As a consequence, linear networks generalize, i.e., reproduce items that were not part of the training set but are consistent with the symmetry of the training set. In contrast, non-linear networks tend to learn individual training items and show associative memory. At the same time, their ability to generalize is limited. A higher degree of generalization is obtained for networks whose activation function contains a linear regime, such as tanh. Our results suggest ANN's ability to generalize - instead of learning items - could be improved by generating a sufficiently big set of elementary operations to represent relations and strongly depends on the applied non-linearity.
Integer Programming for Learning Directed Acyclic Graphs from Non-identifiable Gaussian Models
Authors: Tong Xu, Armeen Taeb, Simge Küçükyavuz, Ali Shojaie
Date: 2024-04-19 00:00:00

We study the problem of learning directed acyclic graphs from continuous observational data, generated according to a linear Gaussian structural equation model. State-of-the-art structure learning methods for this setting have at least one of the following shortcomings: i) they cannot provide optimality guarantees and can suffer from learning sub-optimal models; ii) they rely on the stringent assumption that the noise is homoscedastic, and hence the underlying model is fully identifiable. We overcome these shortcomings and develop a computationally efficient mixed-integer programming framework for learning medium-sized problems that accounts for arbitrary heteroscedastic noise. We present an early stopping criterion under which we can terminate the branch-and-bound procedure to achieve an asymptotically optimal solution and establish the consistency of this approximate solution. In addition, we show via numerical experiments that our method outperforms state-of-the-art algorithms and is robust to noise heteroscedasticity, whereas the performance of some competing methods deteriorates under strong violations of the identifiability assumption. The software implementation of our method is available as the Python package \emph{micodag}.
Benchmarking changepoint detection algorithms on cardiac time series
Authors: Ayse Cakmak, Erik Reinertsen, Shamim Nemati, Gari D. Clifford
Date: 2024-04-16 00:00:00
Tasks: Benchmarking, Change Point Detection, Classification, Time Series

The pattern of state changes in a biomedical time series can be related to health or disease. This work presents a principled approach for selecting a changepoint detection algorithm for a specific task, such as disease classification. Eight key algorithms were compared, and the performance of each algorithm was evaluated as a function of temporal tolerance, noise, and abnormal conduction (ectopy) on realistic artificial cardiovascular time series data. All algorithms were applied to real data (cardiac time series of 22 patients with REM-behavior disorder (RBD) and 15 healthy controls) using the parameters selected on artificial data. Finally, features were derived from the detected changepoints to classify RBD patients from healthy controls using a K-Nearest Neighbors approach. On artificial data, Modified Bayesian Changepoint Detection algorithm provided superior positive predictive value for state change identification while Recursive Mean Difference Maximization (RMDM) achieved the highest true positive rate. For the classification task, features derived from the RMDM algorithm provided the highest leave one out cross validated accuracy of 0.89 and true positive rate of 0.87. Automatically detected changepoints provide useful information about subject's physiological state which cannot be directly observed. However, the choice of change point detection algorithm depends on the nature of the underlying data and the downstream application, such as a classification task. This work represents the first time change point detection algorithms have been compared in a meaningful way and utilized in a classification task, which demonstrates the effect of changepoint algorithm choice on application performance.
Optimized Dynamic Mode Decomposition for Reconstruction and Forecasting of Atmospheric Chemistry Data
Authors: Meghana Velegar, Christoph Keller, J. Nathan Kutz
Date: 2024-04-13 00:00:00

We introduce the optimized dynamic mode decomposition algorithm for constructing an adaptive and computationally efficient reduced order model and forecasting tool for global atmospheric chemistry dynamics. By exploiting a low-dimensional set of global spatio-temporal modes, interpretable characterizations of the underlying spatial and temporal scales can be computed. Forecasting is also achieved with a linear model that uses a linear superposition of the dominant spatio-temporal features. The DMD method is demonstrated on three months of global chemistry dynamics data, showing its significant performance in computational speed and interpretability. We show that the presented decomposition method successfully extracts known major features of atmospheric chemistry, such as summertime surface pollution and biomass burning activities. Moreover, the DMD algorithm allows for rapid reconstruction of the underlying linear model, which can then easily accommodate non-stationary data and changes in the dynamics.
Pneumonia Diagnosis through pixels -- A Deep Learning Model for detection and classification
Authors: Amit Karanth Gurpur, Janani S, Ajeetha B, Brintha Therese A, Rajeswaran Rangasami
Date: 2024-04-15 00:00:00
Tasks: Classification

Manual identification and classification of pneumonia and COVID-19 infection is a cumbersome process that, if delayed can cause irreversible damage to the patient. We have compiled CT scan images from various sources, namely, from the China Consortium of Chest CT Image Investigation (CC-CCII), the Negin Radiology located at Sari in Iran, an open access COVID-19 repository from Havard dataverse, and Sri Ramachandra University, Chennai, India. The images were preprocessed using various methods such as normalization, sharpening, median filter application, binarizing, and cropping to ensure uniformity while training the models. We present an ensemble classification approach using deep learning and machine learning methods to classify patients with the said diseases. Our ensemble model uses pre-trained networks such as ResNet-18 and ResNet-50 for classification and MobileNetV2 for feature extraction. The features from MobileNetV2 are used by the gradient-boosting classifier for the classification of patients. Using ResNet-18, ResNet-50, and the MobileNetV2 aided gradient boosting classifier, we propose an ensemble model with an accuracy of 98 percent on unseen data.
Quantifying seasonal hydrogen storage demands under cost and market uptake uncertainties in energy system transformation pathways
Authors: Felix Frischmuth, Mattis Berghoff, Martin Braun, Philipp Haertel
Date: 2024-04-19 00:00:00

Climate neutrality paradigms put electricity systems at the core of a clean energy supply. At the same time, indirect electrification, with a potential uptake of hydrogen or derived fuel economy, plays a crucial role in decarbonising the energy supply and industrial processes. Besides energy markets coordinating the transition, climate and energy policy targets require fundamental changes and expansions in the energy transmission, import, distribution, and storage infrastructures. While existing studies identify relevant demands for hydrogen, critical decisions involve imports versus domestic fuel production and investments in new or repurposing existing pipeline and storage infrastructure. Linking the pan-European energy system planning model SCOPE SD with the multiperiod European gas market model IMAGINE, the case study analysis and its transformation pathway results indicate extensive network development of hydrogen infrastructure, including expansion beyond refurbished methane infrastructure. However, the ranges of future hydrogen storage costs and market uptake restrictions expose and quantify the uncertainty of its role in Europes transformation. The study finds that rapidly planning the construction of hydrogen storage and pipeline infrastructure is crucial to achieving the required capacity by 2050.
SOAR: Scene-debiasing Open-set Action Recognition
Authors: Yuanhao Zhai, Ziyi Liu, Zhenyu Wu, Yi Wu, Chunluan Zhou, David Doermann, Junsong Yuan, Gang Hua
Date: 2023-09-03 00:00:00
Proceeding: ICCV 2023 1
Tasks: Action Recognition, Decoder, Open Set Action Recognition, Scene Classification

Deep learning models have a risk of utilizing spurious clues to make predictions, such as recognizing actions based on the background scene. This issue can severely degrade the open-set action recognition performance when the testing samples have different scene distributions from the training samples. To mitigate this problem, we propose a novel method, called Scene-debiasing Open-set Action Recognition (SOAR), which features an adversarial scene reconstruction module and an adaptive adversarial scene classification module. The former prevents the decoder from reconstructing the video background given video features, and thus helps reduce the background information in feature learning. The latter aims to confuse scene type classification given video features, with a specific emphasis on the action foreground, and helps to learn scene-invariant information. In addition, we design an experiment to quantify the scene bias. The results indicate that the current open-set action recognizers are biased toward the scene, and our proposed SOAR method better mitigates such bias. Furthermore, our extensive experiments demonstrate that our method outperforms state-of-the-art methods, and the ablation studies confirm the effectiveness of our proposed modules.
BiGCN: Leveraging Cell and Gene Similarities for Single-cell Transcriptome Imputation with Bi-Graph Convolutional Networks
Authors: Yoshitaka Inoue, Ethan Kulman, Rui Kuang
Date: 2024-04-10 00:00:00
Proceeding: bioRxiv 2024 4
Tasks: Imputation

Motivation RNA profiling at the single-cell level is essential for characterizing the molecular activities and functions of individual cells. The current technical limitations of single-cell RNA sequencing (scRNA-seq) technologies can lead to a phenomenon known as “dropout”, where a significant portion of gene expression is not captured. Dropout is particularly prominent in genes with low or sparse expression, greatly impacting the reliability and interpretability of scRNA-seq data. Consequently, various techniques have been developed to estimate missing gene expression using imputation, often by either modeling similarities in gene expression among cells or using gene co-expression, but rarely both.

Results In this study, we introduce a Bi-Graph Convolutional Network (BiGCN), a deep learning method that leverages both cell similarities and gene co-expression to capture cell-type-specific gene co-expression patterns for imputing scRNA-seq data. BiGCN constructs both a cell similarity graph and a gene co-expression graph, and employs them for convolutional smoothing in a dual two-layer Graph Convolutional Networks (GCNs). The embeddings from the two GCNs can subsequently be combined to facilitate the final imputation. BiGCN demonstrates superior performance compared to state-of-the-art imputation methods on both real and simulated scRNA-seq data. Additionally, BiGCN outperforms existing methods when tasked with clustering cells into cell types. We also perform a novel validation using a PBMC scRNA-seq dataset, and this experiment supports that BiGCN’s imputations are more realistic than competing imputation methods. In both the imputation and the cluster tasks, BiGCN consistently outperformed two variants of BiGCN that solely relied on either the gene co-expression graph or cell similarity graph. This indicates that the two graphs offer complimentary information for imputation and cell clustering, underscoring the importance of incorporating both types of information.
Capturing Momentum: Tennis Match Analysis Using Machine Learning and Time Series Theory
Authors: Jingdi Lei, Tianqi Kang, Yuluan Cao, Shiwei Ren
Date: 2024-04-20 00:00:00
Tasks: Feature Importance, Time Series

This paper represents an analysis on the momentum of tennis match. And due to Generalization performance of it, it can be helpful in constructing a system to predict the result of sports game and analyze the performance of player based on the Technical statistics. We First use hidden markov models to predict the momentum which is defined as the performance of players. Then we use Xgboost to prove the significance of momentum. Finally we use LightGBM to evaluate the performance of our model and use SHAP feature importance ranking and weight analysis to find the key points that affect the performance of players.