pwc-archive/papers-with-abstracts huggingface dataset:
Foundation Models for Low-Resource Language Education (Vision Paper)
Authors: Zhaojun Ding, Zhengliang Liu, Hanqi Jiang, Yizhu Gao, Xiaoming Zhai, Tianming Liu, Ninghao Liu
Date: 2024-12-06 00:00:00

Recent studies show that large language models (LLMs) are powerful tools for working with natural language, bringing advances in many areas of computational linguistics. However, these models face challenges when applied to low-resource languages due to limited training data and difficulty in understanding cultural nuances. Research is now focusing on multilingual models to improve LLM performance for these languages. Education in these languages also struggles with a lack of resources and qualified teachers, particularly in underdeveloped regions. Here, LLMs can be transformative, supporting innovative methods like community-driven learning and digital platforms. This paper discusses how LLMs could enhance education for low-resource languages, emphasizing practical applications and benefits.
ChatNVD: Advancing Cybersecurity Vulnerability Assessment with Large Language Models
Authors: Shivansh Chopra, Hussain Ahmad, Diksha Goel, Claudia Szabo
Date: 2024-12-06 00:00:00

The increasing frequency and sophistication of cybersecurity vulnerabilities in software systems underscores the need for more robust and effective vulnerability assessment methods. However, existing approaches often rely on highly technical and abstract frameworks, which hinder understanding and increase the likelihood of exploitation, resulting in severe cyberattacks. In this paper, we introduce ChatNVD, a support tool powered by Large Language Models (LLMs) that leverages the National Vulnerability Database (NVD) to generate accessible, context-rich summaries of software vulnerabilities. We develop three variants of ChatNVD, utilizing three prominent LLMs: GPT-4o Mini by OpenAI, LLaMA 3 by Meta, and Gemini 1.5 Pro by Google. To evaluate their performance, we conduct a comparative evaluation focused on their ability to identify, interpret, and explain software vulnerabilities. Our results demonstrate that GPT-4o Mini outperforms the other models, achieving over 92% accuracy and the lowest error rates, making it the most reliable option for real-world vulnerability assessment.
Formulation of probability theory problem with subtle condition
Authors: Rafayel Petrosyan
Date: 2024-12-05 00:00:00

Problems in probability theory prove to be one of the most challenging for students. Here, we formulate and discuss four related problems in probability theory that proved difficult for first to fourth-year undergraduate students whose first language was not English. These examples emphasize how crucial it is to understand the conditions and requirements of the problems precisely before starting to solve them. We discuss the solutions to those problems in detail, complement them with numerical estimations, and link the conditions in the problems to the logical statements in Python programming language. We also tested two widely used chatbots (GPT-4o and Claude 3.5 Sonnet) by checking their responses to these problems.
Give me Some Hard Questions: Synthetic Data Generation for Clinical QA
Authors: Fan Bai, Keith Harrigian, Joel Stremmel, Hamid Hassanzadeh, Ardavan Saeedi, Mark Dredze
Date: 2024-12-05 00:00:00
Tasks: Question Answering, Question Generation, Question-Generation, Synthetic Data Generation

Clinical Question Answering (QA) systems enable doctors to quickly access patient information from electronic health records (EHRs). However, training these systems requires significant annotated data, which is limited due to the expertise needed and the privacy concerns associated with clinical data. This paper explores generating Clinical QA data using large language models (LLMs) in a zero-shot setting. We find that naive prompting often results in easy questions that do not reflect the complexity of clinical scenarios. To address this, we propose two prompting strategies: 1) instructing the model to generate questions that do not overlap with the input context, and 2) summarizing the input record using a predefined schema to scaffold question generation. Experiments on two Clinical QA datasets demonstrate that our method generates more challenging questions, significantly improving fine-tuning performance over baselines. We compare synthetic and gold data and find a gap between their training efficacy resulting from the quality of synthetically generated answers.
Code generation and runtime techniques for enabling data-efficient deep learning training on GPUs
Authors: Kun Wu
Date: 2024-12-06 00:00:00
Tasks: Code Generation, Deep Learning, GPU

As deep learning models scale, their training cost has surged significantly. Due to both hardware advancements and limitations in current software stacks, the need for data efficiency has risen. Data efficiency refers to the effective hiding of data access latency and the avoidance of unnecessary data movements. Major challenges arise from the growing disparity between GPU memory bandwidth and computational throughput, imminent GPU memory capacity limitations, and inefficiencies in the PyTorch software stack, including a lack of device-specific PCIe transfer optimizations and high-level domain-specific abstractions. To effectively mitigate these data inefficiencies for deep learning training, this dissertation analyzes data inefficiency in representative deep training tasks, specifically in graph neural networks (GNNs) and large language models (LLMs). It then proposes novel runtime and code generation techniques to mitigate these challenges and implements these optimizations seamlessly within the PyTorch stack while maintaining strong programmability and interoperability. First, PyTorch-Direct is devised to incorporate the GPU-centric PCIe data transfer paradigm in PyTorch for GNN training. Next, Hector intermediate representation (IR) and its code generator are proposed to introduce domain-specific high-level abstraction and systematically address memory-intensive performance challenges for relational GNNs. Finally, in LLM training, the throughput has been increasingly constrained by GPU memory capacity. To mitigate this, the SSDTrain offloading framework is designed and implemented. Together, these contributions show that code generation and runtime techniques can systematically mitigate the data management bottlenecks in deep learning training, which stem from the data-intensive nature of workloads and the oversimplification inherent in the deep learning training software stack.
Coal Mine Safety Alert System: Refining BP Neural Network with Genetic Algorithm Optimization
Authors: Jiabin Luo, Hanzhe Pan
Date: 2024-11-21 00:00:00
Tasks: Management

In response to the persistent safety challenges within coal mines, this study proposes a novel approach integrating a three-layer feedforward backpropagation artificial neural network with a genetic algorithm (GA-BP) for establishing a safety early warning system. Focused on a coal mine in Shandong, China, the model's effectiveness is evaluated using relevant data for training and analysis. Results indicate the superiority of the GA-BP model over traditional BP neural networks, offering enhanced capability for identifying potential safety risks promptly. This advancement enables coal mine management to implement timely interventions, ensuring the safety of miners. The findings present valuable insights for engineering applications in similar contexts.
Constructing optimal treatment length strategies to maximize quality-adjusted lifetimes
Authors: Hao Sun, Ashkan Ertefaie, Luke Duttweiler, Brent A. Johnson
Date: 2024-12-06 00:00:00
Tasks: counterfactual, Decision Making

Real-world clinical decision making is a complex process that involves balancing the risks and benefits of treatments. Quality-adjusted lifetime is a composite outcome that combines patient quantity and quality of life, making it an attractive outcome in clinical research. We propose methods for constructing optimal treatment length strategies to maximize this outcome. Existing methods for estimating optimal treatment strategies for survival outcomes cannot be applied to a quality-adjusted lifetime due to induced informative censoring. We propose a weighted estimating equation that adjusts for both confounding and informative censoring. We also propose a nonparametric estimator of the mean counterfactual quality-adjusted lifetime survival curve under a given treatment length strategy, where the weights are estimated using an undersmoothed sieve-based estimator. We show that the estimator is asymptotically linear and provide a data-dependent undersmoothing criterion. We apply our method to obtain the optimal time for percutaneous endoscopic gastrostomy insertion in patients with amyotrophic lateral sclerosis.
Modeling High-Dimensional Dependent Data in the Presence of Many Explanatory Variables and Weak Signals
Authors: Zhaoxing Gao, Ruey S. Tsay
Date: 2024-12-06 00:00:00
Tasks: regression

This article considers a novel and widely applicable approach to modeling high-dimensional dependent data when a large number of explanatory variables are available and the signal-to-noise ratio is low. We postulate that a $p$-dimensional response series is the sum of a linear regression with many observable explanatory variables and an error term driven by some latent common factors and an idiosyncratic noise. The common factors have dynamic dependence whereas the covariance matrix of the idiosyncratic noise can have diverging eigenvalues to handle the situation of low signal-to-noise ratio commonly encountered in applications. The regression coefficient matrix is estimated using penalized methods when the dimensions involved are high. We apply factor modeling to the regression residuals, employ a high-dimensional white noise testing procedure to determine the number of common factors, and adopt a projected Principal Component Analysis when the signal-to-noise ratio is low. We establish asymptotic properties of the proposed method, both for fixed and diverging numbers of regressors, as $p$ and the sample size $T$ approach infinity. Finally, we use simulations and empirical applications to demonstrate the efficacy of the proposed approach in finite samples.
Generalized Recorrupted-to-Recorrupted: Self-Supervised Learning Beyond Gaussian Noise
Authors: Brayan Monroy, Jorge Bacca, Julián Tachella
Date: 2024-12-05 00:00:00
Proceeding: CVPR 2025 1
Tasks: Denoising, Image Restoration, Self-Supervised Learning

Recorrupted-to-Recorrupted (R2R) has emerged as a methodology for training deep networks for image restoration in a self-supervised manner from noisy measurement data alone, demonstrating equivalence in expectation to the supervised squared loss in the case of Gaussian noise. However, its effectiveness with non-Gaussian noise remains unexplored. In this paper, we propose Generalized R2R (GR2R), extending the R2R framework to handle a broader class of noise distribution as additive noise like log-Rayleigh and address the natural exponential family including Poisson and Gamma noise distributions, which play a key role in many applications including low-photon imaging and synthetic aperture radar. We show that the GR2R loss is an unbiased estimator of the supervised loss and that the popular Stein's unbiased risk estimator can be seen as a special case. A series of experiments with Gaussian, Poisson, and Gamma noise validate GR2R's performance, showing its effectiveness compared to other self-supervised methods.
Semiparametric Bayesian Difference-in-Differences
Authors: Christoph Breunig, Ruixuan Liu, Zhengfei Yu
Date: 2024-12-05 00:00:00
Tasks: Bayesian Inference

This paper studies semiparametric Bayesian inference for the average treatment effect on the treated (ATT) within the difference-in-differences (DiD) research design. We propose two new Bayesian methods with frequentist validity. The first one places a standard Gaussian process prior on the conditional mean function of the control group. The second method is a double robust Bayesian procedure that adjusts the prior distribution of the conditional mean function and subsequently corrects the posterior distribution of the resulting ATT. We prove new semiparametric Bernstein-von Mises (BvM) theorems for both proposals. Monte Carlo simulations and an empirical application demonstrate that the proposed Bayesian DiD methods exhibit strong finite-sample performance compared to existing frequentist methods. We also present extensions of the canonical DiD approach, incorporating both the staggered design and the repeated cross-sectional design.
Diff4Steer: Steerable Diffusion Prior for Generative Music Retrieval with Semantic Guidance
Authors: Xuchan Bao, Judith Yue Li, Zhong Yi Wan, Kun Su, Timo Denk, Joonseok Lee, Dima Kuzmin, Fei Sha
Date: 2024-12-06 00:00:00
Tasks: Retrieval

Modern music retrieval systems often rely on fixed representations of user preferences, limiting their ability to capture users' diverse and uncertain retrieval needs. To address this limitation, we introduce Diff4Steer, a novel generative retrieval framework that employs lightweight diffusion models to synthesize diverse seed embeddings from user queries that represent potential directions for music exploration. Unlike deterministic methods that map user query to a single point in embedding space, Diff4Steer provides a statistical prior on the target modality (audio) for retrieval, effectively capturing the uncertainty and multi-faceted nature of user preferences. Furthermore, Diff4Steer can be steered by image or text inputs, enabling more flexible and controllable music discovery combined with nearest neighbor search. Our framework outperforms deterministic regression methods and LLM-based generative retrieval baseline in terms of retrieval and ranking metrics, demonstrating its effectiveness in capturing user preferences, leading to more diverse and relevant recommendations. Listening examples are available at tinyurl.com/diff4steer.
Argumentative Experience: Reducing Confirmation Bias on Controversial Issues through LLM-Generated Multi-Persona Debates
Authors: Li Shi, Houjiang Liu, Yian Wong, Utkarsh Mujumdar, Dan Zhang, Jacek Gwizdka, Matthew Lease
Date: 2024-12-05 00:00:00

Large language models (LLMs) are enabling designers to give life to exciting new user experiences for information access. In this work, we present a system that generates LLM personas to debate a topic of interest from different perspectives. How might information seekers use and benefit from such a system? Can centering information access around diverse viewpoints help to mitigate thorny challenges like confirmation bias in which information seekers over-trust search results matching existing beliefs? How do potential biases and hallucinations in LLMs play out alongside human users who are also fallible and possibly biased? Our study exposes participants to multiple viewpoints on controversial issues via a mixed-methods, within-subjects study. We use eye-tracking metrics to quantitatively assess cognitive engagement alongside qualitative feedback. Compared to a baseline search system, we see more creative interactions and diverse information-seeking with our multi-persona debate system, which more effectively reduces user confirmation bias and conviction toward their initial beliefs. Overall, our study contributes to the emerging design space of LLM-based information access systems, specifically investigating the potential of simulated personas to promote greater exposure to information diversity, emulate collective intelligence, and mitigate bias in information seeking.
Arctic-Embed 2.0: Multilingual Retrieval Without Compromise
Authors: Puxuan Yu, Luke Merrick, Gaurav Nuti, Daniel Campos
Date: 2024-12-03 00:00:00
Tasks: Representation Learning, Retrieval

This paper presents the training methodology of Arctic-Embed 2.0, a set of open-source text embedding models built for accurate and efficient multilingual retrieval. While prior works have suffered from degraded English retrieval quality, Arctic-Embed 2.0 delivers competitive retrieval quality on multilingual and English-only benchmarks, and supports Matryoshka Representation Learning (MRL) for efficient embedding storage with significantly lower compressed quality degradation compared to alternatives. We detail the design and implementation, presenting several important open research questions that arose during model development. We conduct experiments exploring these research questions and include extensive discussion aimed at fostering further discussion in this field.
Epinet for Content Cold Start
Authors: Hong Jun Jeon, Songbin Liu, Yuantong Li, Jie Lyu, Hunter Song, Ji Liu, Peng Wu, Zheqing Zhu
Date: 2024-11-20 00:00:00
Tasks: Recommendation Systems, Thompson Sampling, Uncertainty Quantification

The exploding popularity of online content and its user base poses an evermore challenging matching problem for modern recommendation systems. Unlike other frontiers of machine learning such as natural language, recommendation systems are responsible for collecting their own data. Simply exploiting current knowledge can lead to pernicious feedback loops but naive exploration can detract from user experience and lead to reduced engagement. This exploration-exploitation trade-off is exemplified in the classic multi-armed bandit problem for which algorithms such as upper confidence bounds (UCB) and Thompson sampling (TS) demonstrate effective performance. However, there have been many challenges to scaling these approaches to settings which do not exhibit a conjugate prior structure. Recent scalable approaches to uncertainty quantification via epinets have enabled efficient approximations of Thompson sampling even when the learning model is a complex neural network. In this paper, we demonstrate the first application of epinets to an online recommendation system. Our experiments demonstrate improvements in both user traffic and engagement efficiency on the Facebook Reels online video platform.
NSTRI Global Collaborative Research Data Platform
Authors: Hyeonhoon Lee, Hanseul Kim, Kyungmin Cho, Hyung-Chul Lee
Date: 2024-11-16 00:00:00

The National Strategic Technology Research Institute (NSTRI) Data Platform operated by Seoul National University Hospital (SNUH) addresses the challenge of accessing Korean healthcare data for international research. This platform provides secure access to pseudonymized Korean healthcare data while integrating international datasets, enabling the development of more equitable and generalizable machine learning models. The system features four key AI-powered components: an intelligent data search engine utilizing domain-specific medical embeddings, a Korean-English medical translation system, a comprehensive drug search engine, and an LLM-powered medical research assistant. The platform implements containerized environments within a secure research pod architecture, ensuring data protection while maintaining research efficiency. The platform currently provides access to 10 distinct medical datasets from SNUH, categorized by access permissions and standardized for cross-dataset analysis. This infrastructure enables global collaborative healthcare research while maintaining strict data protection standards.
Posterior Sampling for Random Noise Attenuation via Score-based Generative Models
Authors: Chuangji Meng, Jinghuai Gao, Baohai Wu, Hongling Chen, and Yajun Tian
Date: 2024-11-19 00:00:00
Proceeding: Geophysics 2024 11
Tasks: Denoising, Geophysics, Seismic Inversion

Random noise attenuation is an ill-posed inverse problem with multiple solutions,especially in complicated field noise situations. We present a method to sample stochastic solutions from the posterior distribution of seismic data for a given noisy input. Posterior sampling can be performed by Langevin dynamics with a conditional score function, which can be described as a trained score network (in score-based generative models) plus an analytical expression related to the noise distribution. Each solution from the posterior distribution is reasonable and of high quality. The numerous solutions we obtain may contain underground structural information of interest. We also achieve interactive posterior sampling by automatically estimating a noise level or manually setting it according to the noise level map of the field noise. Experiments on synthetic and field data verify the superiority of our posterior sampling approach.
Take Package as Language: Anomaly Detection Using Transformer
Authors: Jie Huang
Date: 2024-11-15 00:00:00
Tasks: Anomaly Detection, Intrusion Detection, Language Modeling, Language Modelling, Network Intrusion Detection, One-Shot Learning, Supervised Anomaly Detection, Unsupervised Pre-training, Weakly-supervised Anomaly Detection

Network data packet anomaly detection faces numerous challenges, including exploring new anomaly supervision signals, researching weakly supervised anomaly detection, and improving model interpretability. This paper proposes NIDS-GPT, a GPT-based causal language model for network intrusion detection. Unlike previous work, NIDS-GPT innovatively treats each number in the packet as an independent "word" rather than packet fields, enabling a more fine-grained data representation. We adopt an improved GPT-2 model and design special tokenizers and embedding layers to better capture the structure and semantics of network data. NIDS-GPT has good scalability, supports unsupervised pre-training, and enhances model interpretability through attention weight visualization. Experiments on the CICIDS2017 and car-hacking datasets show that NIDS-GPT achieves 100\% accuracy under extreme imbalance conditions, far surpassing traditional methods; it also achieves over 90\% accuracy in one-shot learning. These results demonstrate NIDS-GPT's excellent performance and potential in handling complex network anomaly detection tasks, especially in data-imbalanced and resource-constrained scenarios. The code is available at \url{https://github.com/woshixiaobai2019/nids-gpt.gi
On Models with Power Conservation in Reflective Intelligent Surfaces and their Design Implications
Authors: Robin J. Williams, Pablo Ramirez-Espinosa, Olena Semenovska, Petar Popovski
Date: 2024-12-06 00:00:00

Reconfigurable intelligent surfaces (RISs) are potential enablers of future wireless communications and sensing applications and use-cases. The RIS is envisioned as a dynamically controllable surface that is capable of transforming impinging electromagnetic waves in terms of angles and polarization. Many models has been proposed to predict the wave-transformation capabilities of potential RISs, where power conservation is ensured by enforcing that the scattered power equals the power impinging upon the aperture of the RIS, without considering whether the scattered field adds coherently of destructively with the source field. In effect, this means that power is not conserved, as elaborated in this paper. With the goal of investigating the implications of global and local power conservation in RISs, work considers a single-layer metasurface based RIS. A complete end-to-end communications channel is given through polarizability modeling and conditions for power conservation and channel reciprocity are derived. The implications of the power conservation conditions upon the end-to-end communications channel is analyzed.