pwc-archive/papers-with-abstracts huggingface dataset:
Research on Violent Text Detection System Based on BERT-fasttext Model
Authors: Yongsheng Yang, Xiaoying Wang
Date: 2024-12-21 00:00:00
Tasks: Natural Language Understanding, text-classification, Text Classification, Text Detection

In the digital age of today, the internet has become an indispensable platform for people's lives, work, and information exchange. However, the problem of violent text proliferation in the network environment has arisen, which has brought about many negative effects. In view of this situation, it is particularly important to build an effective system for cutting off violent text. The study of violent text cutting off based on the BERT-fasttext model has significant meaning. BERT is a pre-trained language model with strong natural language understanding ability, which can deeply mine and analyze text semantic information; Fasttext itself is an efficient text classification tool with low complexity and good effect, which can quickly provide basic judgments for text processing. By combining the two and applying them to the system for cutting off violent text, on the one hand, it can accurately identify violent text, and on the other hand, it can efficiently and reasonably cut off the content, preventing harmful information from spreading freely on the network. Compared with the single BERT model and fasttext, the accuracy was improved by 0.7% and 0.8%, respectively. Through this model, it is helpful to purify the network environment, maintain the health of network information, and create a positive, civilized, and harmonious online communication space for netizens, driving the development of social networking, information dissemination, and other aspects in a more benign direction.
InfoTech Assistant : A Multimodal Conversational Agent for InfoTechnology Web Portal Queries
Authors: Sai Surya Gadiraju, Duoduo Liao, Akhila Kudupudi, Santosh Kasula, Charitha Chalasani
Date: 2024-12-21 00:00:00
Tasks: Chatbot, RAG, Retrieval-augmented Generation

This pilot study presents the development of the InfoTech Assistant, a domain-specific, multimodal chatbot engineered to address queries in bridge evaluation and infrastructure technology. By integrating web data scraping, large language models (LLMs), and Retrieval-Augmented Generation (RAG), the InfoTech Assistant provides accurate and contextually relevant responses. Data, including textual descriptions and images, are sourced from publicly available documents on the InfoTechnology website and organized in JSON format to facilitate efficient querying. The architecture of the system includes an HTML-based interface and a Flask back end connected to the Llama 3.1 model via LLM Studio. Evaluation results show approximately 95 percent accuracy on domain-specific tasks, with high similarity scores confirming the quality of response matching. This RAG-enhanced setup enables the InfoTech Assistant to handle complex, multimodal queries, offering both textual and visual information in its responses. The InfoTech Assistant demonstrates strong potential as a dependable tool for infrastructure professionals, delivering high accuracy and relevance in its domain-specific outputs.
Application of Multimodal Large Language Models in Autonomous Driving
Authors: Md Robiul Islam
Date: 2024-12-21 00:00:00
Tasks: Autonomous Driving, Decision Making, Language Modeling, Language Modelling, Large Language Model, Question Answering, Scene Understanding, Visual Question Answering (VQA)

In this era of technological advancements, several cutting-edge techniques are being implemented to enhance Autonomous Driving (AD) systems, focusing on improving safety, efficiency, and adaptability in complex driving environments. However, AD still faces some problems including performance limitations. To address this problem, we conducted an in-depth study on implementing the Multi-modal Large Language Model. We constructed a Virtual Question Answering (VQA) dataset to fine-tune the model and address problems with the poor performance of MLLM on AD. We then break down the AD decision-making process by scene understanding, prediction, and decision-making. Chain of Thought has been used to make the decision more perfectly. Our experiments and detailed analysis of Autonomous Driving give an idea of how important MLLM is for AD.
Inference Scaling vs Reasoning: An Empirical Analysis of Compute-Optimal LLM Problem-Solving
Authors: Marwan AbdElhameed, Pavly Halim
Date: 2024-12-20 00:00:00
Tasks: Computational Efficiency, GSM8K

Recent advances in large language models (LLMs) have predominantly focused on maximizing accuracy and reasoning capabilities, often overlooking crucial computational efficiency considerations. While this approach has yielded impressive accuracy improvements, it has led to methods that may be impractical for real-world deployment due to computational overhead and latency constraints. This paper investigates the potential synergy between reasoning enhancement and computational efficiency by analyzing the integration of two contrasting approaches: Quiet-STaR (Self-Taught Reasoner) and REBASE (REward BAlanced SEarch). Through comprehensive empirical analysis using the Mistral-7B model on the GSM8K dataset, we demonstrate that while each method excels in its primary objective-Quiet-STaR achieving superior accuracy (32.03%) despite high computational cost (554.66s runtime, 12.73T FLOPs), and REBASE providing exceptional efficiency (8.47s runtime, 2.35T FLOPs) while maintaining baseline-comparable accuracy (10.94%)-their integration reveals fundamental challenges in reconciling reasoning depth with computational efficiency. The combined approach unexpectedly results in degraded performance (9.38% accuracy, 143.66s runtime), highlighting critical insights about the complex interplay between reasoning enhancement and efficiency optimization in LLMs. Our findings illuminate the need for novel architectures and algorithms specifically designed to bridge the gap between these competing objectives, while providing concrete directions for future research in compute-efficient reasoning methods.
Adversarial Robustness through Dynamic Ensemble Learning
Authors: Hetvi Waghela, Jaydip Sen, Sneha Rakshit
Date: 2024-12-20 00:00:00
Tasks: Adversarial Attack, Adversarial Robustness, Diversity, Ensemble Learning

Adversarial attacks pose a significant threat to the reliability of pre-trained language models (PLMs) such as GPT, BERT, RoBERTa, and T5. This paper presents Adversarial Robustness through Dynamic Ensemble Learning (ARDEL), a novel scheme designed to enhance the robustness of PLMs against such attacks. ARDEL leverages the diversity of multiple PLMs and dynamically adjusts the ensemble configuration based on input characteristics and detected adversarial patterns. Key components of ARDEL include a meta-model for dynamic weighting, an adversarial pattern detection module, and adversarial training with regularization techniques. Comprehensive evaluations using standardized datasets and various adversarial attack scenarios demonstrate that ARDEL significantly improves robustness compared to existing methods. By dynamically reconfiguring the ensemble to prioritize the most robust models for each input, ARDEL effectively reduces attack success rates and maintains higher accuracy under adversarial conditions. This work contributes to the broader goal of developing more secure and trustworthy AI systems for real-world NLP applications, offering a practical and scalable solution to enhance adversarial resilience in PLMs.
Quantum Time-Series Learning with Evolutionary Algorithms
Authors: Vignesh Anantharamakrishnan, Márcio M. Taddei
Date: 2024-12-23 00:00:00
Tasks: Evolutionary Algorithms, Time Series, Time Series Forecasting

Variational quantum circuits have arisen as an important method in quantum computing. A crucial step of it is parameter optimization, which is typically tackled through gradient-descent techniques. We advantageously explore instead the use of evolutionary algorithms for such optimization, specifically for time-series forecasting. We perform a comparison, for diverse instances of real-world data, between gradient-descent parameter optimization and covariant-matrix adaptation evolutionary strategy. We observe that gradient descent becomes permanently trapped in local minima that have been avoided by evolutionary algorithms in all tested datasets, reaching up to a six-fold decrease in prediction error. Finally, the combined use of evolutionary and gradient-based techniques is explored, aiming at retaining advantages of both. The results are particularly applicable in scenarios sensitive to gains in accuracy.
Tiered Acquisition for Constrained Bayesian Optimization: An Application to Analog Circuits
Authors: Ria Rashid, Abhishek Gupta
Date: 2024-12-23 00:00:00
Tasks: Bayesian Optimization

Analog circuit design can be considered as an optimization problem with the targeted circuit specifications as constraints. When stringent circuit specifications are considered, it is desired to have an optimization methodology that adapts well to heavily constrained search spaces. To this end, we propose a novel Bayesian optimization algorithm with a tiered ensemble of acquisition functions and demonstrate its considerable application potential for analog circuit design automation. Our method is the first to introduce the concept of multiple dominance among acquisition functions, allowing the search for the optimal solutions to be effectively bounded \emph{within} the predicted set of feasible solutions in a constrained search space. This has resulted in a significant reduction in constraint violations by the candidate solutions, leading to better-optimized designs within tight computational budgets. The methodology is validated in gain and area optimization of a two-stage Miller compensated operational amplifier in a 65 nm technology. In comparison to robust baselines and state-of-the-art algorithms, this method reduces constraint violations by up to 38% and improves the target objective by up to 43%. The source code of our algorithm is made available at https://github.com/riarashid/TRACE.
A diversity-enhanced genetic algorithm for efficient exploration of parameter spaces
Authors: Jonas Wessén, Eliel Camargo-Molina
Date: 2024-12-22 00:00:00
Tasks: Diversity, Efficient Exploration

We present a Python package together with a practical guide for the implementation of a lightweight diversity-enhanced genetic algorithm (GA) approach for the exploration of multi-dimensional parameter spaces. Searching a parameter space for regions with desirable properties, e.g. compatibility with experimental data, poses a type of optimization problem wherein the focus lies on pinpointing all "good enough" solutions, rather than a single "best solution". Our approach dramatically outperforms random scans and other GA-based implementations in this aspect. We validate the effectiveness of our approach by applying it to a particle physics problem, showcasing its ability to identify promising parameter points in isolated, viable regions meeting experimental constraints. The companion Python package is applicable to optimization problems beyond those considered in this work, including scanning over discrete parameters (categories). A detailed guide for its usage is provided.
Adaptive Calibration: A Unified Conversion Framework of Spiking Neural Network
Authors: Ziqing Wang, Yuetong Fang, Jiahang Cao, Hongwei Ren, Renjing Xu
Date: 2024-12-18 00:00:00
Tasks: object-detection, Object Detection

Spiking Neural Networks (SNNs) are seen as an energy-efficient alternative to traditional Artificial Neural Networks (ANNs), but the performance gap remains a challenge. While this gap is narrowing through ANN-to-SNN conversion, substantial computational resources are still needed, and the energy efficiency of converted SNNs cannot be ensured. To address this, we present a unified training-free conversion framework that significantly enhances both the performance and efficiency of converted SNNs. Inspired by the biological nervous system, we propose a novel Adaptive-Firing Neuron Model (AdaFire), which dynamically adjusts firing patterns across different layers to substantially reduce the Unevenness Error - the primary source of error of converted SNNs within limited inference timesteps. We further introduce two efficiency-enhancing techniques: the Sensitivity Spike Compression (SSC) technique for reducing spike operations, and the Input-aware Adaptive Timesteps (IAT) technique for decreasing latency. These methods collectively enable our approach to achieve state-of-the-art performance while delivering significant energy savings of up to 70.1%, 60.3%, and 43.1% on CIFAR-10, CIFAR-100, and ImageNet datasets, respectively. Extensive experiments across 2D, 3D, event-driven classification tasks, object detection, and segmentation tasks, demonstrate the effectiveness of our method in various domains. The code is available at: https://github.com/bic-L/burst-ann2snn.
Bivariate Matrix-valued Linear Regression (BMLR): Finite-sample performance under Identifiability and Sparsity Assumptions
Authors: Nayel Bettache
Date: 2024-12-23 00:00:00
Tasks: Denoising

This study explores the estimation of parameters in a matrix-valued linear regression model, where the $T$ responses $(Y_t)_{t=1}^T \in \mathbb{R}^{n \times p}$ and predictors $(X_t)_{t=1}^T \in \mathbb{R}^{m \times q}$ satisfy the relationship $Y_t = A^* X_t B^* + E_t$ for all $t = 1, \ldots, T$. In this model, $A^* \in \mathbb{R}_+^{n \times m}$ has $L_1$-normalized rows, $B^* \in \mathbb{R}^{q \times p}$, and $(E_t)_{t=1}^T$ are independent noise matrices following a matrix Gaussian distribution. The primary objective is to estimate the unknown parameters $A^*$ and $B^*$ efficiently. We propose explicit optimization-free estimators and establish non-asymptotic convergence rates to quantify their performance. Additionally, we extend our analysis to scenarios where $A^*$ and $B^*$ exhibit sparse structures. To support our theoretical findings, we conduct numerical simulations that confirm the behavior of the estimators, particularly with respect to the impact of the dimensions $n, m, p, q$, and the sample size $T$ on finite-sample performances. We complete the simulations by investigating the denoising performances of our estimators on noisy real-world images.
Bayesian penalized empirical likelihood and Markov Chain Monte Carlo sampling
Authors: Jinyuan Chang, Cheng Yong Tang, Yuanzheng Zhu
Date: 2024-12-23 00:00:00

In this study, we introduce a novel methodological framework called Bayesian Penalized Empirical Likelihood (BPEL), designed to address the computational challenges inherent in empirical likelihood (EL) approaches. Our approach has two primary objectives: (i) to enhance the inherent flexibility of EL in accommodating diverse model conditions, and (ii) to facilitate the use of well-established Markov Chain Monte Carlo (MCMC) sampling schemes as a convenient alternative to the complex optimization typically required for statistical inference using EL. To achieve the first objective, we propose a penalized approach that regularizes the Lagrange multipliers, significantly reducing the dimensionality of the problem while accommodating a comprehensive set of model conditions. For the second objective, our study designs and thoroughly investigates two popular sampling schemes within the BPEL context. We demonstrate that the BPEL framework is highly flexible and efficient, enhancing the adaptability and practicality of EL methods. Our study highlights the practical advantages of using sampling techniques over traditional optimization methods for EL problems, showing rapid convergence to the global optima of posterior distributions and ensuring the effective resolution of complex statistical inference challenges.
Decoupled Functional Central Limit Theorems for Two-Time-Scale Stochastic Approximation
Authors: Yuze Han, Xiang Li, Jiadong Liang, Zhihua Zhang
Date: 2024-12-22 00:00:00

In two-time-scale stochastic approximation (SA), two iterates are updated at different rates, governed by distinct step sizes, with each update influencing the other. Previous studies have demonstrated that the convergence rates of the error terms for these updates depend solely on their respective step sizes, a property known as decoupled convergence. However, a functional version of this decoupled convergence has not been explored. Our work fills this gap by establishing decoupled functional central limit theorems for two-time-scale SA, offering a more precise characterization of its asymptotic behavior. To achieve these results, we leverage the martingale problem approach and establish tightness as a crucial intermediate step. Furthermore, to address the interdependence between different time scales, we introduce an innovative auxiliary sequence to eliminate the primary influence of the fast-time-scale update on the slow-time-scale update.
Kernel Methods for the Approximation of the Eigenfunctions of the Koopman Operator
Authors: Jonghyeon Lee, Boumediene Hamzi, Boya Hou, Houman Owhadi, Gabriele Santin, Umesh Vaidya
Date: 2024-12-21 00:00:00

The Koopman operator provides a linear framework to study nonlinear dynamical systems. Its spectra offer valuable insights into system dynamics, but the operator can exhibit both discrete and continuous spectra, complicating direct computations. In this paper, we introduce a kernel-based method to construct the principal eigenfunctions of the Koopman operator without explicitly computing the operator itself. These principal eigenfunctions are associated with the equilibrium dynamics, and their eigenvalues match those of the linearization of the nonlinear system at the equilibrium point. We exploit the structure of the principal eigenfunctions by decomposing them into linear and nonlinear components. The linear part corresponds to the left eigenvector of the system's linearization at the equilibrium, while the nonlinear part is obtained by solving a partial differential equation (PDE) using kernel methods. Our approach avoids common issues such as spectral pollution and spurious eigenvalues, which can arise in previous methods. We demonstrate the effectiveness of our algorithm through numerical examples.
Transport Quasi-Monte Carlo
Authors: Sifan Liu
Date: 2024-12-21 00:00:00
Tasks: Bayesian Inference

Quasi-Monte Carlo (QMC) is a powerful method for evaluating high-dimensional integrals. However, its use is typically limited to distributions where direct sampling is straightforward, such as the uniform distribution on the unit hypercube or the Gaussian distribution. For general target distributions with potentially unnormalized densities, leveraging the low-discrepancy property of QMC to improve accuracy remains challenging. We propose training a transport map to push forward the uniform distribution on the unit hypercube to approximate the target distribution. Inspired by normalizing flows, the transport map is constructed as a composition of simple, invertible transformations. To ensure that RQMC achieves its superior error rate, the transport map must satisfy specific regularity conditions. We introduce a flexible parametrization for the transport map that not only meets these conditions but is also expressive enough to model complex distributions. Our theoretical analysis establishes that the proposed transport QMC estimator achieves faster convergence rates than standard Monte Carlo, under mild and easily verifiable growth conditions on the integrand. Numerical experiments confirm the theoretical results, demonstrating the effectiveness of the proposed method in Bayesian inference tasks.
Post-hoc Interpretability Illumination for Scientific Interaction Discovery
Authors: Ling Zhang, Zhichao Hou, Tingxiang Ji, Yuanyuan Xu, Runze Li
Date: 2024-12-20 00:00:00
Tasks: Decision Making, scientific discovery

Model interpretability and explainability have garnered substantial attention in recent years, particularly in decision-making applications. However, existing interpretability tools often fall short in delivering satisfactory performance due to limited capabilities or efficiency issues. To address these challenges, we propose a novel post-hoc method: Iterative Kings' Forests (iKF), designed to uncover complex multi-order interactions among variables. iKF iteratively selects the next most important variable, the "King", and constructs King's Forests by placing it at the root node of each tree to identify variables that interact with the "King". It then generates ranked short lists of important variables and interactions of varying orders. Additionally, iKF provides inference metrics to analyze the patterns of the selected interactions and classify them into one of three interaction types: Accompanied Interaction, Synergistic Interaction, and Hierarchical Interaction. Extensive experiments demonstrate the strong interpretive power of our proposed iKF, highlighting its great potential for explainable modeling and scientific discovery across diverse scientific fields.
Towards scientific discovery with dictionary learning: Extracting biological concepts from microscopy foundation models
Authors: Konstantin Donhauser, Kristina Ulicna, Gemma Elyse Moran, Aditya Ravuri, Kian Kenyon-Dean, Cian Eastwood, Jason Hartford
Date: 2024-12-20 00:00:00
Tasks: Dictionary Learning, scientific discovery

Dictionary learning (DL) has emerged as a powerful interpretability tool for large language models. By extracting known concepts (e.g., Golden-Gate Bridge) from human-interpretable data (e.g., text), sparse DL can elucidate a model's inner workings. In this work, we ask if DL can also be used to discover unknown concepts from less human-interpretable scientific data (e.g., cell images), ultimately enabling modern approaches to scientific discovery. As a first step, we use DL algorithms to study microscopy foundation models trained on multi-cell image data, where little prior knowledge exists regarding which high-level concepts should arise. We show that sparse dictionaries indeed extract biologically-meaningful concepts such as cell type and genetic perturbation type. We also propose a new DL algorithm, Iterative Codebook Feature Learning~(ICFL), and combine it with a pre-processing step that uses PCA whitening from a control dataset. In our experiments, we demonstrate that both ICFL and PCA improve the selectivity of extracted features compared to TopK sparse autoencoders.
Utilizing Causal Network Markers to Identify Tipping Points ahead of Critical Transition
Authors: Shirui Bian, Zezhou Wang, Siyang Leng, Wei Lin, Jifan Shi
Date: 2024-12-19 00:00:00

Early-warning signals of delicate design are always used to predict critical transitions in complex systems, which makes it possible to render the systems far away from the catastrophic state by introducing timely interventions. Traditional signals including the dynamical network biomarker (DNB), based on statistical properties such as variance and autocorrelation of nodal dynamics, overlook directional interactions and thus have limitations in capturing underlying mechanisms and simultaneously sustaining robustness against noise perturbations. This paper therefore introduces a framework of causal network markers (CNMs) by incorporating causality indicators, which reflect the directional influence between variables. Actually, to detect and identify the tipping points ahead of critical transition, two markers are designed: CNM-GC for linear causality and CNM-TE for non-linear causality, as well as a functional representation of different causality indicators and a clustering technique to verify the system's dominant group. Through demonstrations using benchmark models and real-world datasets of epileptic seizure, the framework of CNMs shows higher predictive power and accuracy than the traditional DNB indicator. It is believed that, due to the versatility and scalability, the CNMs are suitable for comprehensively evaluating the systems. The most possible direction for application includes the identification of tipping points in clinical disease.