pwc-archive/papers-with-abstracts huggingface dataset:
A Variational Computational-based Framework for Unsteady Incompressible Flows
Authors: H. Sababha, A. Elmaradny, H. Taha, M. Daqaq
Date: 2024-12-07 00:00:00

Advancements in computational fluid mechanics have largely relied on Newtonian frameworks, particularly through the direct simulation of Navier-Stokes equations. In this work, we propose an alternative computational framework that employs variational methods, specifically by leveraging the principle of minimum pressure gradient, which turns the fluid mechanics problem into a minimization problem whose solution can be used to predict the flow field in unsteady incompressible viscous flows. This method exhibits two particulary intriguing properties. First, it circumvents the chronic issues of pressure-velocity coupling in incompressible flows, which often dominates the computational cost in computational fluid dynamics (CFD). Second, this method eliminates the reliance on unphysical assumptions at the outflow boundary, addressing another longstanding challenge in CFD. We apply this framework to three benchmark examples across a range of Reynolds numbers: (i) unsteady flow field in a lid-driven cavity, (ii) Poiseuille flow, and (iii) flow past a circular cylinder. The minimization framework is carried out using a physics-informed neural network (PINN), which integrates the underlying physical principles directly into the training of the model. The results from the proposed method are validated against high-fidelity CFD simulations, showing an excellent agreement. Comparison of the proposed variational method to the conventional method, wherein PINNs is directly applied to solve Navier-Stokes Equations, reveals that the proposed method outperforms conventional PINNs in terms of both convergence rate and time, demonstrating its potential for solving complex fluid mechanics problems.
Confidence Diagram of Nonparametric Ranking for Uncertainty Assessment in Large Language Models Evaluation
Authors: Zebin Wang, Yi Han, Ethan X. Fang, Lan Wang, Junwei Lu
Date: 2024-12-07 00:00:00
Tasks: Language Modeling, Language Modelling, Large Language Model, valid

We consider the inference for the ranking of large language models (LLMs). Alignment arises as a significant challenge to mitigate hallucinations in the use of LLMs. Ranking LLMs has proven to be an effective tool to improve alignment based on the best-of-$N$ policy. In this paper, we propose a new inferential framework for hypothesis testing among the ranking for language models. Our framework is based on a nonparametric contextual ranking framework designed to assess large language models' domain-specific expertise, leveraging nonparametric scoring methods to account for their sensitivity to the prompts. To characterize the combinatorial complexity of the ranking, we introduce a novel concept of confidence diagram, which leverages a Hasse diagram to represent the entire confidence set of rankings by a single directed graph. We show the validity of the proposed confidence diagram by advancing the Gaussian multiplier bootstrap theory to accommodate the supremum of independent empirical processes that are not necessarily identically distributed. Extensive numerical experiments conducted on both synthetic and real data demonstrate that our approach offers valuable insight into the evaluation for the performance of different LLMs across various medical domains.
SplaXBERT: Leveraging Mixed Precision Training and Context Splitting for Question Answering
Authors: Zhu Yufan, Hao Zeyu, Li Siqi, Niu Boqian
Date: 2024-12-07 00:00:00
Tasks: Question Answering

SplaXBERT, built on ALBERT-xlarge with context-splitting and mixed precision training, achieves high efficiency in question-answering tasks on lengthy texts. Tested on SQuAD v1.1, it attains an Exact Match of 85.95% and an F1 Score of 92.97%, outperforming traditional BERT-based models in both accuracy and resource efficiency.
Flex Attention: A Programming Model for Generating Optimized Attention Kernels
Authors: Juechu Dong, Boyuan Feng, Driss Guessous, Yanbo Liang, Horace He
Date: 2024-12-07 00:00:00

Over the past 7 years, attention has become one of the most important primitives in deep learning. The primary approach to optimize attention is FlashAttention, which fuses the operation together, drastically improving both the runtime and the memory consumption. However, the importance of FlashAttention combined with its monolithic nature poses a problem for researchers aiming to try new attention variants -- a "software lottery". This problem is exacerbated by the difficulty of writing efficient fused attention kernels, resisting traditional compiler-based approaches. We introduce FlexAttention, a novel compiler-driven programming model that allows implementing the majority of attention variants in a few lines of idiomatic PyTorch code. We demonstrate that many existing attention variants (e.g. Alibi, Document Masking, PagedAttention, etc.) can be implemented via FlexAttention, and that we achieve competitive performance compared to these handwritten kernels. Finally, we demonstrate how FlexAttention allows for easy composition of attention variants, solving the combinatorial explosion of attention variants.
AI-powered Digital Twin of the Ocean: Reliable Uncertainty Quantification for Real-time Wave Height Prediction with Deep Ensemble
Authors: Dongeon Lee, Sunwoong Yang, Jae-Won Oh, Su-Gil Cho, SangHyuk Kim, Namwoo Kang
Date: 2024-12-07 00:00:00
Tasks: Uncertainty Quantification

Environmental pollution and fossil fuel depletion have prompted the need for renewable energy-based power generation. However, its stability is often challenged by low energy density and non-stationary conditions. Wave energy converters (WECs), in particular, need reliable real-time wave height prediction to address these issues caused by irregular wave patterns, which can lead to the inefficient and unstable operation of WECs. In this study, we propose an AI-powered reliable real-time wave height prediction model that integrates long short-term memory (LSTM) networks for temporal prediction with deep ensemble (DE) for robust uncertainty quantification (UQ), ensuring high accuracy and reliability. To further enhance the reliability, uncertainty calibration is applied, which has proven to significantly improve the quality of the quantified uncertainty. Using real operational data from an oscillating water column-wave energy converter (OWC-WEC) system in Jeju, South Korea, the model achieves notable accuracy (R2 > 0.9), while increasing uncertainty quality by over 50% through simple calibration technique. Furthermore, a comprehensive parametric study is conducted to explore the effects of key model hyperparameters, offering valuable guidelines for diverse operational scenarios, characterized by differences in wavelength, amplitude, and period. These results demonstrate the model's capability to deliver reliable predictions, facilitating digital twin of the ocean.
Multi-Objective Alignment of Large Language Models Through Hypervolume Maximization
Authors: Subhojyoti Mukherjee, Anusha Lalitha, Sailik Sengupta, Aniket Deshmukh, Branislav Kveton
Date: 2024-12-06 00:00:00
Tasks: Hallucination

Multi-objective alignment from human feedback (MOAHF) in large language models (LLMs) is a challenging problem as human preferences are complex, multifaceted, and often conflicting. Recent works on MOAHF considered a-priori multi-objective optimization (MOO), where human preferences are known at training or inference time. In contrast, when human preferences are unknown or difficult to quantify, a natural approach is to cover the Pareto front by multiple diverse solutions. We propose an algorithm HaM for learning diverse LLM policies that maximizes their hypervolume. This is the first application of a-posteriori MOO to MOAHF. HaM is computationally and space efficient, and empirically superior across objectives such as harmlessness, helpfulness, humor, faithfulness, and hallucination, on various datasets.
Granular Ball K-Class Twin Support Vector Classifier
Authors: M. A. Ganaie, Vrushank Ahire, Anouck Girard
Date: 2024-12-06 00:00:00
Tasks: Binary Classification, Classification, Fault Diagnosis, Multi-class Classification

This paper introduces the Granular Ball K-Class Twin Support Vector Classifier (GB-TWKSVC), a novel multi-class classification framework that combines Twin Support Vector Machines (TWSVM) with granular ball computing. The proposed method addresses key challenges in multi-class classification by utilizing granular ball representation for improved noise robustness and TWSVM's non-parallel hyperplane architecture solves two smaller quadratic programming problems, enhancing efficiency. Our approach introduces a novel formulation that effectively handles multi-class scenarios, advancing traditional binary classification methods. Experimental evaluation on diverse benchmark datasets shows that GB-TWKSVC significantly outperforms current state-of-the-art classifiers in both accuracy and computational performance. The method's effectiveness is validated through comprehensive statistical tests and complexity analysis. Our work advances classification algorithms by providing a mathematically sound framework that addresses the scalability and robustness needs of modern machine learning applications. The results demonstrate GB-TWKSVC's broad applicability across domains including pattern recognition, fault diagnosis, and large-scale data analytics, establishing it as a valuable addition to the classification algorithm landscape.
Diversity Over Quantity: A Lesson From Few Shot Relation Classification
Authors: Amir DN Cohen, Shauli Ravfogel, Shaltiel Shmidman, Yoav Goldberg
Date: 2024-12-06 00:00:00
Tasks: Diversity, Few-Shot Learning, Few-Shot Relation Classification, Relation, Relation Classification

In few-shot relation classification (FSRC), models must generalize to novel relations with only a few labeled examples. While much of the recent progress in NLP has focused on scaling data size, we argue that diversity in relation types is more crucial for FSRC performance. In this work, we demonstrate that training on a diverse set of relations significantly enhances a model's ability to generalize to unseen relations, even when the overall dataset size remains fixed. We introduce REBEL-FS, a new FSRC benchmark that incorporates an order of magnitude more relation types than existing datasets. Through systematic experiments, we show that increasing the diversity of relation types in the training data leads to consistent gains in performance across various few-shot learning scenarios, including high-negative settings. Our findings challenge the common assumption that more data alone leads to better performance and suggest that targeted data curation focused on diversity can substantially reduce the need for large-scale datasets in FSRC.
DART-Eval: A Comprehensive DNA Language Model Evaluation Benchmark on Regulatory DNA
Authors: Aman Patel, Arpita Singhal, Austin Wang, Anusri Pampari, Maya Kasowski, Anshul Kundaje
Date: 2024-12-06 00:00:00
Tasks: counterfactual, Language Model Evaluation, Language Modeling, Language Modelling

Recent advances in self-supervised models for natural language, vision, and protein sequences have inspired the development of large genomic DNA language models (DNALMs). These models aim to learn generalizable representations of diverse DNA elements, potentially enabling various genomic prediction, interpretation and design tasks. Despite their potential, existing benchmarks do not adequately assess the capabilities of DNALMs on key downstream applications involving an important class of non-coding DNA elements critical for regulating gene activity. In this study, we introduce DART-Eval, a suite of representative benchmarks specifically focused on regulatory DNA to evaluate model performance across zero-shot, probed, and fine-tuned scenarios against contemporary ab initio models as baselines. Our benchmarks target biologically meaningful downstream tasks such as functional sequence feature discovery, predicting cell-type specific regulatory activity, and counterfactual prediction of the impacts of genetic variants. We find that current DNALMs exhibit inconsistent performance and do not offer compelling gains over alternative baseline models for most tasks, while requiring significantly more computational resources. We discuss potentially promising modeling, data curation, and evaluation strategies for the next generation of DNALMs. Our code is available at https://github.com/kundajelab/DART-Eval.
No Free Lunch From Random Feature Ensembles
Authors: Benjamin S. Ruben, William L. Tong, Hamza Tahir Chaudhry, Cengiz Pehlevan
Date: 2024-12-06 00:00:00
Tasks: regression

Given a budget on total model size, one must decide whether to train a single, large neural network or to combine the predictions of many smaller networks. We study this trade-off for ensembles of random-feature ridge regression models. We prove that when a fixed number of trainable parameters are partitioned among $K$ independently trained models, $K=1$ achieves optimal performance, provided the ridge parameter is optimally tuned. We then derive scaling laws which describe how the test risk of an ensemble of regression models decays with its total size. We identify conditions on the kernel and task eigenstructure under which ensembles can achieve near-optimal scaling laws. Training ensembles of deep convolutional neural networks on CIFAR-10 and a transformer architecture on C4, we find that a single large network outperforms any ensemble of networks with the same total number of parameters, provided the weight decay and feature-learning strength are tuned to their optimal values.
Knowledge-Based Deep Learning for Time-Efficient Inverse Dynamics
Authors: Shuhao Ma, Yu Cao, Ian D. Robertson, Chaoyang Shi, Jindong Liu, Zhi-Qiang Zhang
Date: 2024-12-06 00:00:00
Tasks: Deep Learning

Accurate understanding of muscle activation and muscle forces plays an essential role in neuro-rehabilitation and musculoskeletal disorder treatments. Computational musculoskeletal modeling has been widely used as a powerful non-invasive tool to estimate them through inverse dynamics using static optimization, but the inherent computational complexity results in time-consuming analysis. In this paper, we propose a knowledge-based deep learning framework for time-efficient inverse dynamic analysis, which can predict muscle activation and muscle forces from joint kinematic data directly while not requiring any label information during model training. The Bidirectional Gated Recurrent Unit (BiGRU) neural network is selected as the backbone of our model due to its proficient handling of time-series data. Prior physical knowledge from forward dynamics and pre-selected inverse dynamics based physiological criteria are integrated into the loss function to guide the training of neural networks. Experimental validations on two datasets, including one benchmark upper limb movement dataset and one self-collected lower limb movement dataset from six healthy subjects, are performed. The experimental results have shown that the selected BiGRU architecture outperforms other neural network models when trained using our specifically designed loss function, which illustrates the effectiveness and robustness of the proposed framework.
Tabular data generation with tensor contraction layers and transformers
Authors: Aníbal Silva, André Restivo, Moisés Santos, Carlos Soares
Date: 2024-12-06 00:00:00
Tasks: Density Estimation, Tabular Data Generation

Generative modeling for tabular data has recently gained significant attention in the Deep Learning domain. Its objective is to estimate the underlying distribution of the data. However, estimating the underlying distribution of tabular data has its unique challenges. Specifically, this data modality is composed of mixed types of features, making it a non-trivial task for a model to learn intra-relationships between them. One approach to address mixture is to embed each feature into a continuous matrix via tokenization, while a solution to capture intra-relationships between variables is via the transformer architecture. In this work, we empirically investigate the potential of using embedding representations on tabular data generation, utilizing tensor contraction layers and transformers to model the underlying distribution of tabular data within Variational Autoencoders. Specifically, we compare four architectural approaches: a baseline VAE model, two variants that focus on tensor contraction layers and transformers respectively, and a hybrid model that integrates both techniques. Our empirical study, conducted across multiple datasets from the OpenML CC18 suite, compares models over density estimation and Machine Learning efficiency metrics. The main takeaway from our results is that leveraging embedding representations with the help of tensor contraction layers improves density estimation metrics, albeit maintaining competitive performance in terms of machine learning efficiency.
BadGPT-4o: stripping safety finetuning from GPT models
Authors: Ekaterina Krupkina, Dmitrii Volkov
Date: 2024-12-06 00:00:00

We show a version of Qi et al. 2023's simple fine-tuning poisoning technique strips GPT-4o's safety guardrails without degrading the model. The BadGPT attack matches best white-box jailbreaks on HarmBench and StrongREJECT. It suffers no token overhead or performance hits common to jailbreaks, as evaluated on tinyMMLU and open-ended generations. Despite having been known for a year, this attack remains easy to execute.
Training Large Language Models to Reason in a Continuous Latent Space
Authors: Shibo Hao, Sainbayar Sukhbaatar, DiJia Su, Xian Li, Zhiting Hu, Jason Weston, Yuandong Tian
Date: 2024-12-09 00:00:00
Tasks: Logical Reasoning

Large language models (LLMs) are restricted to reason in the "language space", where they typically express the reasoning process with a chain-of-thought (CoT) to solve a complex reasoning problem. However, we argue that language space may not always be optimal for reasoning. For example, most word tokens are primarily for textual coherence and not essential for reasoning, while some critical tokens require complex planning and pose huge challenges to LLMs. To explore the potential of LLM reasoning in an unrestricted latent space instead of using natural language, we introduce a new paradigm Coconut (Chain of Continuous Thought). We utilize the last hidden state of the LLM as a representation of the reasoning state (termed "continuous thought"). Rather than decoding this into a word token, we feed it back to the LLM as the subsequent input embedding directly in the continuous space. Experiments show that Coconut can effectively augment the LLM on several reasoning tasks. This novel latent reasoning paradigm leads to emergent advanced reasoning patterns: the continuous thought can encode multiple alternative next reasoning steps, allowing the model to perform a breadth-first search (BFS) to solve the problem, rather than prematurely committing to a single deterministic path like CoT. Coconut outperforms CoT in certain logical reasoning tasks that require substantial backtracking during planning, with fewer thinking tokens during inference. These findings demonstrate the promise of latent reasoning and offer valuable insights for future research.
AutoDCWorkflow: LLM-based Data Cleaning Workflow Auto-Generation and Benchmark
Authors: Lan Li, Liri Fang, Vetle I. Torvik
Date: 2024-12-09 00:00:00
Tasks: Missing Values

We investigate the reasoning capabilities of large language models (LLMs) for automatically generating data-cleaning workflows. To evaluate LLMs' ability to complete data-cleaning tasks, we implemented a pipeline for LLM-based Auto Data Cleaning Workflow (AutoDCWorkflow), prompting LLMs on data cleaning operations to repair three types of data quality issues: duplicates, missing values, and inconsistent data formats. Given a dirty table and a purpose (expressed as a query), this pipeline generates a minimal, clean table sufficient to address the purpose and the data cleaning workflow used to produce the table. The planning process involves three main LLM-driven components: (1) Select Target Columns: Identifies a set of target columns related to the purpose. (2) Inspect Column Quality: Assesses the data quality for each target column and generates a Data Quality Report as operation objectives. (3) Generate Operation & Arguments: Predicts the next operation and arguments based on the data quality report results. Additionally, we propose a data cleaning benchmark to evaluate the capability of LLM agents to automatically generate workflows that address data cleaning purposes of varying difficulty levels. The benchmark comprises the annotated datasets as a collection of purpose, raw table, clean table, data cleaning workflow, and answer set. In our experiments, we evaluated three LLMs that auto-generate purpose-driven data cleaning workflows. The results indicate that LLMs perform well in planning and generating data-cleaning workflows without the need for fine-tuning.
GEAR: A Simple GENERATE, EMBED, AVERAGE AND RANK Approach for Unsupervised Reverse Dictionary
Authors: Fatemah Almeman, Luis Espinosa-Anke
Date: 2024-12-09 00:00:00
Tasks: Reverse Dictionary, Sentence, Sentence Embeddings

Reverse Dictionary (RD) is the task of obtaining the most relevant word or set of words given a textual description or dictionary definition. Effective RD methods have applications in accessibility, translation or writing support systems. Moreover, in NLP research we find RD to be used to benchmark text encoders at various granularities, as it often requires word, definition and sentence embeddings. In this paper, we propose a simple approach to RD that leverages LLMs in combination with embedding models. Despite its simplicity, this approach outperforms supervised baselines in well studied RD datasets, while also showing less over-fitting. We also conduct a number of experiments on different dictionaries and analyze how different styles, registers and target audiences impact the quality of RD systems. We conclude that, on average, untuned embeddings alone fare way below an LLM-only baseline (although they are competitive in highly technical dictionaries), but are crucial for boosting performance in combined methods.
Anchoring Bias in Large Language Models: An Experimental Study
Authors: Jiaxu Lou, Yifan Sun
Date: 2024-12-09 00:00:00

Large Language Models (LLMs) like GPT-4 and Gemini have significantly advanced artificial intelligence by enabling machines to generate and comprehend human-like text. Despite their impressive capabilities, LLMs are not immune to limitations, including various biases. While much research has explored demographic biases, the cognitive biases in LLMs have not been equally scrutinized. This study delves into anchoring bias, a cognitive bias where initial information disproportionately influences judgment. Utilizing an experimental dataset, we examine how anchoring bias manifests in LLMs and verify the effectiveness of various mitigation strategies. Our findings highlight the sensitivity of LLM responses to biased hints. At the same time, our experiments show that, to mitigate anchoring bias, one needs to collect hints from comprehensive angles to prevent the LLMs from being anchored to individual pieces of information, while simple algorithms such as Chain-of-Thought, Thoughts of Principles, Ignoring Anchor Hints, and Reflection are not sufficient.