pwc-archive/papers-with-abstracts huggingface dataset:
Machine Learning for Everyone: Simplifying Healthcare Analytics with BigQuery ML
Authors: Mohammad Amir Salari, Bahareh Rahmani
Date: 2025-02-10 00:00:00
Tasks: Diabetes Prediction

Machine learning (ML) is transforming healthcare by enabling predictive analytics, personalized treatments, and improved patient outcomes. However, traditional ML workflows require specialized skills, infrastructure, and resources, limiting accessibility for many healthcare professionals. This paper explores how Google Cloud's BigQuery ML simplifies the development and deployment of ML models using SQL, reducing technical barriers. Through a case study on diabetes prediction using the Diabetes Health Indicators Dataset, we evaluate three predictive models: Logistic Regression, Boosted Tree, and Deep Neural Network (DNN). Our results demonstrate that the Boosted Tree model achieves the highest performance, making it highly effective for diabetes prediction. This study highlights BigQuery ML's role in democratizing machine learning by providing a scalable, efficient, and accessible solution for healthcare analytics.
Related Knowledge Perturbation Matters: Rethinking Multiple Pieces of Knowledge Editing in Same-Subject
Authors: Zenghao Duan, Wenbin Duan, Zhiyi Yin, Yinghan Shen, Shaoling Jing, Jie Zhang, HuaWei Shen, Xueqi Cheng
Date: 2025-02-08 00:00:00
Tasks: knowledge editing

Knowledge editing has become a promising approach for efficiently and precisely updating knowledge embedded in large language models (LLMs). In this work, we focus on Same-Subject Editing, which involves modifying multiple attributes of a single entity to ensure comprehensive and consistent updates to entity-centric knowledge. Through preliminary observation, we identify a significant challenge: Current state-of-the-art editing methods struggle when tasked with editing multiple related knowledge pieces for the same subject. To address the lack of relevant editing data for identical subjects in traditional benchmarks, we introduce the $\text{S}^2\text{RKE}$(Same-Subject Related Knowledge Editing) benchmark. Our extensive experiments reveal that only mainstream locate-then-edit methods, such as ROME and MEMIT, exhibit "related knowledge perturbation," where subsequent edits interfere with earlier ones. Further analysis reveals that these methods over-rely on subject information, neglecting other critical factors, resulting in reduced editing effectiveness.
Semi-Supervised Vision-Centric 3D Occupancy World Model for Autonomous Driving
Authors: Xiang Li, Pengfei Li, Yupeng Zheng, Wei Sun, Yan Wang, Yilun Chen
Date: 2025-02-11 00:00:00
Tasks: Attribute, Autonomous Driving, Motion Planning

Understanding world dynamics is crucial for planning in autonomous driving. Recent methods attempt to achieve this by learning a 3D occupancy world model that forecasts future surrounding scenes based on current observation. However, 3D occupancy labels are still required to produce promising results. Considering the high annotation cost for 3D outdoor scenes, we propose a semi-supervised vision-centric 3D occupancy world model, PreWorld, to leverage the potential of 2D labels through a novel two-stage training paradigm: the self-supervised pre-training stage and the fully-supervised fine-tuning stage. Specifically, during the pre-training stage, we utilize an attribute projection head to generate different attribute fields of a scene (e.g., RGB, density, semantic), thus enabling temporal supervision from 2D labels via volume rendering techniques. Furthermore, we introduce a simple yet effective state-conditioned forecasting module to recursively forecast future occupancy and ego trajectory in a direct manner. Extensive experiments on the nuScenes dataset validate the effectiveness and scalability of our method, and demonstrate that PreWorld achieves competitive performance across 3D occupancy prediction, 4D occupancy forecasting and motion planning tasks.
Mask-Enhanced Autoregressive Prediction: Pay Less Attention to Learn More
Authors: Xialie Zhuang, Zhikai Jia, Jianjin Li, Zhenyu Zhang, Li Shen, Zheng Cao, Shiwei Liu
Date: 2025-02-11 00:00:00
Tasks: Decoder, Information Retrieval, Language Modeling, Language Modelling, Masked Language Modeling, Retrieval

Large Language Models (LLMs) are discovered to suffer from accurately retrieving key information. To address this, we propose Mask-Enhanced Autoregressive Prediction (MEAP), a simple yet effective training paradigm that seamlessly integrates Masked Language Modeling (MLM) into Next-Token Prediction (NTP) to enhance the latter's in-context retrieval capabilities. Specifically, MEAP first randomly masks a small fraction of input tokens and then directly performs the standard next-token prediction autoregressive using a decoder-only Transformer. MEAP eliminates the need for bidirectional attention or encoder-decoder architectures for MLM, incurring no additional computational overhead during pre-training or inference. Intensive experiments demonstrate that MEAP substantially outperforms NTP on key information retrieval and long-context reasoning tasks, while performing on par or better on commonsense reasoning tasks. The benefits of MEAP also extend to supervised fine-tuning, where it shows remarkable advantages in lost-in-the-middle scenarios, outperforming NTP by 11.77 percentage points. Our analysis indicates that MEAP's effectiveness arises from its ability to promote more distinguishable attention scores by concentrating on a reduced set of non-masked tokens. This mechanism improves the model's focus on task-relevant signals while mitigating the influence of peripheral context. These findings position MEAP as a promising training paradigm for large language models.
WebChecker: A Versatile EVL Plugin for Validating HTML Pages with Bootstrap Frameworks
Authors: Milind Cherukuri
Date: 2025-02-11 00:00:00

WebChecker is a plugin for Epsilon Validation Language (EVL), designed to validate both static and dynamic HTML pages utilizing frameworks like Bootstrap. By employing configurable EVL constraints, WebChecker enforces implicit rules governing HTML and CSS frameworks. The effectiveness of the plugin is demonstrated through its application on Bootstrap, the widely adopted HTML, CSS, and JavaScript framework. WebChecker comes with a set of EVL constraints to assess Bootstrap based web pages. To substantiate our claims, I present an illustrative example featuring two solutions that effectively enforce implicit rules.
CTR-Driven Advertising Image Generation with Multimodal Large Language Models
Authors: Xingye Chen, Wei Feng, Zhenbang Du, Weizhen Wang, Yanyin Chen, Haohan Wang, Linkai Liu, Yaoyu Li, Jinyuan Zhao, Yu Li, Zheng Zhang, Jingjing Lv, Junjie Shen, Zhangang Lin, Jingping Shao, Yuanjie Shao, Xinge You, Changxin Gao, Nong Sang
Date: 2025-02-05 00:00:00
Tasks: Image Generation, Reinforcement Learning (RL)

In web data, advertising images are crucial for capturing user attention and improving advertising effectiveness. Most existing methods generate background for products primarily focus on the aesthetic quality, which may fail to achieve satisfactory online performance. To address this limitation, we explore the use of Multimodal Large Language Models (MLLMs) for generating advertising images by optimizing for Click-Through Rate (CTR) as the primary objective. Firstly, we build targeted pre-training tasks, and leverage a large-scale e-commerce multimodal dataset to equip MLLMs with initial capabilities for advertising image generation tasks. To further improve the CTR of generated images, we propose a novel reward model to fine-tune pre-trained MLLMs through Reinforcement Learning (RL), which can jointly utilize multimodal features and accurately reflect user click preferences. Meanwhile, a product-centric preference optimization strategy is developed to ensure that the generated background content aligns with the product characteristics after fine-tuning, enhancing the overall relevance and effectiveness of the advertising images. Extensive experiments have demonstrated that our method achieves state-of-the-art performance in both online and offline metrics. Our code and pre-trained models are publicly available at: https://github.com/Chenguoz/CAIG.
Multimodal Cognitive Reframing Therapy via Multi-hop Psychotherapeutic Reasoning
Authors: Subin Kim, Hoonrae Kim, Heejin Do, Gary Geunbae Lee
Date: 2025-02-08 00:00:00

Previous research has revealed the potential of large language models (LLMs) to support cognitive reframing therapy; however, their focus was primarily on text-based methods, often overlooking the importance of non-verbal evidence crucial in real-life therapy. To alleviate this gap, we extend the textual cognitive reframing to multimodality, incorporating visual clues. Specifically, we present a new dataset called Multi Modal-Cognitive Support Conversation (M2CoSC), which pairs each GPT-4-generated dialogue with an image that reflects the virtual client's facial expressions. To better mirror real psychotherapy, where facial expressions lead to interpreting implicit emotional evidence, we propose a multi-hop psychotherapeutic reasoning approach that explicitly identifies and incorporates subtle evidence. Our comprehensive experiments with both LLMs and vision-language models (VLMs) demonstrate that the VLMs' performance as psychotherapists is significantly improved with the M2CoSC dataset. Furthermore, the multi-hop psychotherapeutic reasoning method enables VLMs to provide more thoughtful and empathetic suggestions, outperforming standard prompting methods.
Adaptive Sampling and Joint Semantic-Channel Coding under Dynamic Channel Environment
Authors: Zhiyuan Qi, Yulong Feng, Zhijin Qin
Date: 2025-02-11 00:00:00

Deep learning enabled semantic communications are attracting extensive attention. However, most works normally ignore the data acquisition process and suffer from robustness issues under dynamic channel environment. In this paper, we propose an adaptive joint sampling-semantic-channel coding (Adaptive-JSSCC) framework. Specifically, we propose a semantic-aware sampling and reconstruction method to optimize the number of samples dynamically for each region of the images. According to semantic significance, we optimize sampling matrices for each region of the most individually and obtain a semantic sampling ratio distribution map shared with the receiver. Through the guidance of the map, high-quality reconstruction is achieved. Meanwhile, attention-based channel adaptive module (ACAM) is designed to overcome the neural network model mismatch between the training and testing channel environment during sampling-reconstruction and encoding-decoding. To this end, signal-to-noise ratio (SNR) is employed as an extra parameter input to integrate and reorganize intermediate characteristics. Simulation results show that the proposed Adaptive-JSSCC effectively reduces the amount of data acquisition without degrading the reconstruction performance in comparison to the state-of-the-art, and it is highly adaptable and adjustable to dynamic channel environments.
UniZyme: A Unified Protein Cleavage Site Predictor Enhanced with Enzyme Active-Site Knowledge
Authors: Chenao Li, Shuo Yan, Enyan Dai
Date: 2025-02-10 00:00:00

Enzyme-catalyzed protein cleavage is essential for many biological functions. Accurate prediction of cleavage sites can facilitate various applications such as drug development, enzyme design, and a deeper understanding of biological mechanisms. However, most existing models are restricted to an individual enzyme, which neglects shared knowledge of enzymes and fails generalize to novel enzymes. Thus, we introduce a unified protein cleavage site predictor named UniZyme, which can generalize across diverse enzymes. To enhance the enzyme encoding for the protein cleavage site prediction, UniZyme employs a novel biochemically-informed model architecture along with active-site knowledge of proteolytic enzymes. Extensive experiments demonstrate that UniZyme achieves high accuracy in predicting cleavage sites across a range of proteolytic enzymes, including unseen enzymes. The code is available in https://anonymous.4open.science/r/UniZyme-4A67.
Leveraging GPT-4o Efficiency for Detecting Rework Anomaly in Business Processes
Authors: Mohammad Derakhshan, Paolo Ceravolo, Fatemeh Mohammadi
Date: 2025-02-10 00:00:00
Tasks: Anomaly Detection

This paper investigates the effectiveness of GPT-4o-2024-08-06, one of the Large Language Models (LLM) from OpenAI, in detecting business process anomalies, with a focus on rework anomalies. In our study, we developed a GPT-4o-based tool capable of transforming event logs into a structured format and identifying reworked activities within business event logs. The analysis was performed on a synthetic dataset designed to contain rework anomalies but free of loops. To evaluate the anomaly detection capabilities of GPT 4o-2024-08-06, we used three prompting techniques: zero-shot, one-shot, and few-shot. These techniques were tested on different anomaly distributions, namely normal, uniform, and exponential, to identify the most effective approach for each case. The results demonstrate the strong performance of GPT-4o-2024-08-06. On our dataset, the model achieved 96.14% accuracy with one-shot prompting for the normal distribution, 97.94% accuracy with few-shot prompting for the uniform distribution, and 74.21% accuracy with few-shot prompting for the exponential distribution. These results highlight the model's potential as a reliable tool for detecting rework anomalies in event logs and how anomaly distribution and prompting strategy influence the model's performance.
The Differences Between Direct Alignment Algorithms are a Blur
Authors: Alexey Gorbatovski, Boris Shaposhnikov, Viacheslav Sinii, Alexey Malakhov, Daniil Gavrilov
Date: 2025-02-03 00:00:00
Tasks: Language Modeling, Language Modelling, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Direct Alignment Algorithms (DAAs) offer a simpler way to language model alignment than traditional RLHF by directly optimizing policies. While DAAs differ in their use of SFT (one-stage vs. two-stage), the scalar scores within their objectives (likelihood vs. odds ratios), and ranking objectives (pairwise vs. pointwise), the critical factors for performance remain underexplored. We provide a systematic comparative analysis. We first show that one-stage methods (e.g. ORPO, ASFT) underperform compared to two-stage approaches. However, we demonstrate that adapting them to a two-stage setup with an explicit SFT phase can improve their performance. Further, introducing and tuning a unifying $\beta$ parameter within this two-stage framework boosts their performence (e.g., AlpacaEval 2: $+13.45$ ORPO, $+8.27$ ASFT), matching established methods like DPO and enabling fair comparisons. Our comprehensive analysis reveals that the choice between pairwise and pointwise objectives is the primary determinant of alignment success, rather than the specific scalar score (e.g., policy-reference ratio vs. odds ratio) employed. We provide empirical evidence suggesting this stems from how these objectives interact with prompt-specific biases. These findings underscore the need for nuanced evaluations in DAA research to avoid oversimplified claims of superiority.
Fact-or-Fair: A Checklist for Behavioral Testing of AI Models on Fairness-Related Queries
Authors: Jen-tse Huang, Yuhang Yan, Linqi Liu, Yixin Wan, Wenxuan Wang, Kai-Wei Chang, Michael R. Lyu
Date: 2025-02-09 00:00:00
Tasks: Diversity, Fairness, World Knowledge

The generation of incorrect images, such as depictions of people of color in Nazi-era uniforms by Gemini, frustrated users and harmed Google's reputation, motivating us to investigate the relationship between accurately reflecting factuality and promoting diversity and equity. In this study, we focus on 19 real-world statistics collected from authoritative sources. Using these statistics, we develop a checklist comprising objective and subjective queries to analyze behavior of large language models (LLMs) and text-to-image (T2I) models. Objective queries assess the models' ability to provide accurate world knowledge. In contrast, the design of subjective queries follows a key principle: statistical or experiential priors should not be overgeneralized to individuals, ensuring that models uphold diversity. These subjective queries are based on three common human cognitive errors that often result in social biases. We propose metrics to assess factuality and fairness, and formally prove the inherent trade-off between these two aspects. Results show that GPT-4o and DALL-E 3 perform notably well among six LLMs and four T2I models. Our code is publicly available at https://github.com/uclanlp/Fact-or-Fair.
OpenGrok: Enhancing SNS Data Processing with Distilled Knowledge and Mask-like Mechanisms
Authors: Lumen AI, Zaozhuang No. 28 Middle School, Shihao Ji, Zihui Song, Fucheng Zhong, Jisen Jia, Zhaobo Wu, Zheyi Cao, Tianhao Xu
Date: 2025-02-11 00:00:00
Tasks: Knowledge Distillation, MMLU, Text-To-SQL

This report details Lumen Labs' novel approach to processing Social Networking Service (SNS) data. We leverage knowledge distillation, specifically a simple distillation method inspired by DeepSeek-R1's CoT acquisition, combined with prompt hacking, to extract valuable training data from the Grok model. This data is then used to fine-tune a Phi-3-mini model, augmented with a mask-like mechanism specifically designed for handling the nuances of SNS data. Our method demonstrates state-of-the-art (SOTA) performance on several SNS data processing tasks, outperforming existing models like Grok, Phi-3, and GPT-4. We provide a comprehensive analysis of our approach, including mathematical formulations, engineering details, ablation studies, and comparative evaluations.
Sequential Stochastic Combinatorial Optimization Using Hierarchal Reinforcement Learning
Authors: Xinsong Feng, Zihan Yu, Yanhai Xiong, Haipeng Chen
Date: 2025-02-08 00:00:00
Tasks: Combinatorial Optimization, Computational Efficiency, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Reinforcement learning (RL) has emerged as a promising tool for combinatorial optimization (CO) problems due to its ability to learn fast, effective, and generalizable solutions. Nonetheless, existing works mostly focus on one-shot deterministic CO, while sequential stochastic CO (SSCO) has rarely been studied despite its broad applications such as adaptive influence maximization (IM) and infectious disease intervention. In this paper, we study the SSCO problem where we first decide the budget (e.g., number of seed nodes in adaptive IM) allocation for all time steps, and then select a set of nodes for each time step. The few existing studies on SSCO simplify the problems by assuming a uniformly distributed budget allocation over the time horizon, yielding suboptimal solutions. We propose a generic hierarchical RL (HRL) framework called wake-sleep option (WS-option), a two-layer option-based framework that simultaneously decides adaptive budget allocation on the higher layer and node selection on the lower layer. WS-option starts with a coherent formulation of the two-layer Markov decision processes (MDPs), capturing the interdependencies between the two layers of decisions. Building on this, WS-option employs several innovative designs to balance the model's training stability and computational efficiency, preventing the vicious cyclic interference issue between the two layers. Empirical results show that WS-option exhibits significantly improved effectiveness and generalizability compared to traditional methods. Moreover, the learned model can be generalized to larger graphs, which significantly reduces the overhead of computational resources.
Q-MARL: A GRAPH-BASED SOLUTION FOR LARGE-SCALE MULTI-AGENT REINFORCEMENT LEARNING INSPIRED BY QUANTUM CHEMISTRY
Authors: Kha Vo
Date: 2025-02-13 00:00:00
Proceeding: khavo.ai 2025 2
Tasks: Multi-agent Reinforcement Learning

Inspired by a graph-based technique for [doing what] in quantum chemistry, we present Q-MARL, a completely decentralised learning architecture that supports very large-scale multi-agent rein- forcement learning scenarios without the need for strong assumptions like common rewards or agent order. The key is to treat each agent as relative to its surrounding agents in an environment that is presumed to change dynamically. Hence, in each time step, an agent is the centre of its own neighbourhood and also a neighbour to many other agents. Each role is formulated as a sub-graph, and each sub-graph is used as a training sample. A message-passing neural network supports full-scale vertex and edge interaction within a local neighbourhood, while a pa- rameter governing the depth of the sub-graphs eases the training burden. During testing, an agent’s actions are locally ensembled across all the sub-graphs that contain it, resulting in robust decisions. Where other approaches struggle to manage 50 agents, Q-MARL can easily marshal thousands. A detailed theoretical analysis proves improvement and con- vergence, and simulations with the typical collaborative and competitive scenarios show dramatically faster training speeds and reduced training losses.
An Analysis of Novel Money Laundering Data Using Heterogeneous Graph Isomorphism Networks. FinCEN Files Case Study
Authors: Filip Wójcik
Date: 2024-07-26 00:00:00
Proceeding: Econometrics. Advances in Applied Data Analysis. 2024 7
Tasks: Graph Neural Network, Knowledge Graphs, Link Prediction

Aim: This study aimed to develop and apply the novel HexGIN (Heterogeneous extension for Graph Isomorphism Network) model to the FinCEN Files case data and compare its performance with existing solutions, such as the SAGE-based graph neural network and Multi-Layer Perceptron (MLP), to demonstrate its potential advantages in the field of anti-money laundering systems (AML).

Methodology: The research employed the FinCEN Files case data to develop and apply the HexGIN model in a beneficiary prediction task for a suspicious transactions graph. The model's performance was compared with the existing solutions in a series of cross-validation experiments.

Results: The experimental results on the cross-validation data and test dataset indicate the potential advantages of HexGIN over the existing solutions, such as MLP and Graph SAGE. The proposed model outperformed other algorithms in terms of F1 score, precision, and ROC AUC in both training and testing phases.

Implications and recommendations: The findings demonstrate the potential of heterogeneous graph neural networks and their highly expressive architectures, such as GIN, in AML. Further research is needed, in particular to combine the proposed model with other existing algorithms and test the solution on various money-laundering datasets.

Originality/value: Unlike many AML studies that rely on synthetic or undisclosed data sources, this research was based on a publicly available, real, heterogeneous transaction dataset, being part of a larger investigation. The results indicate a promising direction for the development of modern hybrid AML tools for analysing suspicious transactions; based on heterogeneous graph networks capable of handling various types of entities and their connections.