pwc-archive/papers-with-abstracts huggingface dataset:
NeuroComparatives: Neuro-Symbolic Distillation of Comparative Knowledge
Authors: Phillip Howard, Junlin Wang, Vasudev Lal, Gadi Singer, Yejin Choi, Swabha Swayamdipta
Date: 2023-05-08 00:00:00
Tasks: Knowledge Distillation, valid, World Knowledge

Comparative knowledge (e.g., steel is stronger and heavier than styrofoam) is an essential component of our world knowledge, yet understudied in prior literature. In this paper, we harvest the dramatic improvements in knowledge capabilities of language models into a large-scale comparative knowledge base. While the ease of acquisition of such comparative knowledge is much higher from extreme-scale models like GPT-4, compared to their considerably smaller and weaker counterparts such as GPT-2, not even the most powerful models are exempt from making errors. We thus ask: to what extent are models at different scales able to generate valid and diverse comparative knowledge? We introduce NeuroComparatives, a novel framework for comparative knowledge distillation overgenerated from language models such as GPT-variants and LLaMA, followed by stringent filtering of the generated knowledge. Our framework acquires comparative knowledge between everyday objects, producing a corpus of up to 8.8M comparisons over 1.74M entity pairs - 10X larger and 30% more diverse than existing resources. Moreover, human evaluations show that NeuroComparatives outperform existing resources in terms of validity (up to 32% absolute improvement). Our acquired NeuroComparatives leads to performance improvements on five downstream tasks. We find that neuro-symbolic manipulation of smaller models offers complementary benefits to the currently dominant practice of prompting extreme-scale language models for knowledge distillation.
Genomic Materials Design: CALculation of PHAse Dynamics
Authors: G. B Olson, Z. K. Liu
Date: 2023-05-08 00:00:00
Tasks: Management

The CALPHAD system of fundamental phase-level databases, now known as the Materials Genome, has enabled a mature technology of computational materials design and qualification that has already met the acceleration goals of the national Materials Genome Initiative. As first commercialized by QuesTek Innovations, the methodology combines efficient genomic-level parametric design of new material composition and process specifications with multidisciplinary simulation-based forecasting of manufacturing variation, integrating efficient uncertainty management. Recent projects demonstrated under the multi-institutional CHiMaD Design Center notably include novel alloys designed specifically for the new technology of additive manufacturing. With the proven success of the CALPHAD-based Materials Genome technology, current university research emphasizes new methodologies for affordable accelerated expansion of more accurate CALPHAD databases. Rapid adoption of these new capabilities by US apex corporations has compressed the materials design and development cycle to under 2 years, enabling a new "materials concurrency" integrated into a new level of concurrent engineering supporting an unprecedented level of manufacturing innovation.
Solving Linear Inverse Problems using Higher-Order Annealed Langevin Diffusion
Authors: Nicolas Zilberstein, Ashutosh Sabharwal, Santiago Segarra
Date: 2023-05-08 00:00:00

We propose a solution for linear inverse problems based on higher-order Langevin diffusion. More precisely, we propose pre-conditioned second-order and third-order Langevin dynamics that provably sample from the posterior distribution of our unknown variables of interest while being computationally more efficient than their first-order counterpart and the non-conditioned versions of both dynamics. Moreover, we prove that both pre-conditioned dynamics are well-defined and have the same unique invariant distributions as the non-conditioned cases. We also incorporate an annealing procedure that has the double benefit of further accelerating the convergence of the algorithm and allowing us to accommodate the case where the unknown variables are discrete. Numerical experiments in two different tasks in communications (MIMO symbol detection and channel estimation) and in three tasks for images showcase the generality of our method and illustrate the high performance achieved relative to competing approaches (including learning-based ones) while having comparable or lower computational complexity.
Error Analysis of Kernel/GP Methods for Nonlinear and Parametric PDEs
Authors: Pau Batlle, Yifan Chen, Bamdad Hosseini, Houman Owhadi, Andrew M Stuart
Date: 2023-05-08 00:00:00

We introduce a priori Sobolev-space error estimates for the solution of nonlinear, and possibly parametric, PDEs using Gaussian process and kernel based methods. The primary assumptions are: (1) a continuous embedding of the reproducing kernel Hilbert space of the kernel into a Sobolev space of sufficient regularity; and (2) the stability of the differential operator and the solution map of the PDE between corresponding Sobolev spaces. The proof is articulated around Sobolev norm error estimates for kernel interpolants and relies on the minimizing norm property of the solution. The error estimates demonstrate dimension-benign convergence rates if the solution space of the PDE is smooth enough. We illustrate these points with applications to high-dimensional nonlinear elliptic PDEs and parametric PDEs. Although some recent machine learning methods have been presented as breaking the curse of dimensionality in solving high-dimensional PDEs, our analysis suggests a more nuanced picture: there is a trade-off between the regularity of the solution and the presence of the curse of dimensionality. Therefore, our results are in line with the understanding that the curse is absent when the solution is regular enough.
Improving Implicit Feedback-Based Recommendation through Multi-Behavior Alignment
Authors: Xin Xin, Xiangyuan Liu, Hanbing Wang, Pengjie Ren, Zhumin Chen, Jiahuan Lei, Xinlei Shi, Hengliang Luo, Joemon Jose, Maarten de Rijke, Zhaochun Ren
Date: 2023-05-09 00:00:00
Tasks: Denoising, Open-Ended Question Answering, Recommendation Systems, Transfer Learning

Recommender systems that learn from implicit feedback often use large volumes of a single type of implicit user feedback, such as clicks, to enhance the prediction of sparse target behavior such as purchases. Using multiple types of implicit user feedback for such target behavior prediction purposes is still an open question. Existing studies that attempted to learn from multiple types of user behavior often fail to: (i) learn universal and accurate user preferences from different behavioral data distributions, and (ii) overcome the noise and bias in observed implicit user feedback. To address the above problems, we propose multi-behavior alignment (MBA), a novel recommendation framework that learns from implicit feedback by using multiple types of behavioral data. We conjecture that multiple types of behavior from the same user (e.g., clicks and purchases) should reflect similar preferences of that user. To this end, we regard the underlying universal user preferences as a latent variable. The variable is inferred by maximizing the likelihood of multiple observed behavioral data distributions and, at the same time, minimizing the Kullback-Leibler divergence (KL-divergence) between user models learned from auxiliary behavior (such as clicks or views) and the target behavior separately. MBA infers universal user preferences from multi-behavior data and performs data denoising to enable effective knowledge transfer. We conduct experiments on three datasets, including a dataset collected from an operational e-commerce platform. Empirical results demonstrate the effectiveness of our proposed method in utilizing multiple types of behavioral data to enhance the prediction of the target behavior.
Learning Personalized Page Content Ranking Using Customer Representation
Authors: Xin Shen, Yan Zhao, Sujan Perera, Yujia Liu, Jinyun Yan, Mitchell Goodman
Date: 2023-05-09 00:00:00

On E-commerce stores, there are rich recommendation content to help shoppers shopping more efficiently. However given numerous products, it's crucial to select most relevant content to reduce the burden of information overload. We introduced a content ranking service powered by a linear causal bandit algorithm to rank and select content for each shopper under each context. The algorithm mainly leverages aggregated customer behavior features, and ignores single shopper level past activities. We study the problem of inferring shoppers interest from historical activities. We propose a deep learning based bandit algorithm that incorporates historical shopping behavior, customer latent shopping goals, and the correlation between customers and content categories. This model produces more personalized content ranking measured by 12.08% nDCG lift.
Popularity Debiasing from Exposure to Interaction in Collaborative Filtering
Authors: YuanHao Liu, Qi Cao, HuaWei Shen, Yunfan Wu, Shuchang Tao, Xueqi Cheng
Date: 2023-05-09 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems

Recommender systems often suffer from popularity bias, where popular items are overly recommended while sacrificing unpopular items. Existing researches generally focus on ensuring the number of recommendations exposure of each item is equal or proportional, using inverse propensity weighting, causal intervention, or adversarial training. However, increasing the exposure of unpopular items may not bring more clicks or interactions, resulting in skewed benefits and failing in achieving real reasonable popularity debiasing. In this paper, we propose a new criterion for popularity debiasing, i.e., in an unbiased recommender system, both popular and unpopular items should receive Interactions Proportional to the number of users who Like it, namely IPL criterion. Under the guidance of the criterion, we then propose a debiasing framework with IPL regularization term which is theoretically shown to achieve a win-win situation of both popularity debiasing and recommendation performance. Experiments conducted on four public datasets demonstrate that when equipping two representative collaborative filtering models with our framework, the popularity bias is effectively alleviated while maintaining the recommendation performance.
Autumn: A Scalable Read Optimized LSM-tree based Key-Value Stores with Fast Point and Range Read Speed
Authors: Fuheng Zhao, Zach Miller, Leron Reznikov, Divyakant Agrawal, Amr El Abbadi
Date: 2023-05-08 00:00:00

The Log Structured Merge Trees (LSM-tree) based key-value stores are widely used in many storage systems to support a variety of operations such as updates, point reads, and range reads. Traditionally, LSM-tree's merge policy organizes data into multiple levels of exponentially increasing capacity to support high-speed writes. However, we contend that the traditional merge policies are not optimized for reads. In this work, we present Autumn, a scalable and read optimized LSM-tree based key-value stores with minimal point and range read cost. The key idea in improving the read performance is to dynamically adjust the capacity ratio between two adjacent levels as more data are stored. As a result, smaller levels gradually increase their capacities and merge more often. In particular, the point and range read cost improves from the previous best known $O(logN)$ complexity to $O(\sqrt{logN})$ in Autumn by applying the novel Garnering merge policy. While Garnering merge policy optimizes for both point reads and range reads, it maintains high performance for updates. Moreover, to further improve the update costs, Autumn uses a small amount of bounded space of DRAM to pin/keep the first level of LSM-tree. We implemented Autumn on top of LevelDB and experimentally showcases the gain in performance for real world workloads.
Fairness-Aware Recommendation of Information Curators
Authors: Ziwei Zhu, Jianling Wang, Yin Zhang, James Caverlee
Date: 2018-09-09 00:00:00
Tasks: Fairness

This paper highlights our ongoing efforts to create effective information curator recommendation models that can be personalized for individual users, while maintaining important fairness properties. Concretely, we introduce the problem of information curator recommendation, provide a high-level overview of a fairness-aware recommender, and introduce some preliminary experimental evidence over a real-world Twitter dataset. We conclude with some thoughts on future directions.
Enhancing Suno's Bark Text-to-Speech Model: Addressing Limitations Through Meta's Encodec and Pre-Trained Hubert
Authors: Devin Schumacher, Francis LaBounty Jr.
Date: 2023-04-18 00:00:00
Proceeding: Social Science Research Network (SSRN) 2023 4
Tasks: Audio Generation, Expressive Speech Synthesis, Speech Synthesis, text-to-speech, Text to Speech, Text-To-Speech Synthesis, Voice Cloning

Bark, a transformer-based text-to-audio model by Suno, generates highly realistic, multilingual speech as well as other audio, including music, background noise, and simple sound effects. While this model has shown promising results, its generative nature can lead to deviations in the output based on provided prompts. In this paper, we propose a novel approach to improve the performance of Bark by leveraging Meta's encodec to extract audio codebooks and employing a pre-trained HuBert model with a linear projection head to generate semantic tokens that better match the source audio.

Our method involves extracting discrete tokens from the audio codebooks using Meta's encodec and saving the fine and coarse prompts. We then use the transcript of the source audio to generate semantic tokens from the original Bark model. However, this process has limitations due to the lack of access to the wav2vec model and its associated k-means used in the original training. To overcome this, we adapt a pre-trained HuBert model with a linear projection head, training it to output tokens in the same embedding space as the unavailable model.

By incorporating these enhancements, we aim to address the limitations of Bark's generative capabilities, ultimately leading to more accurate text-to-speech outputs. Our work contributes to the ongoing development of advanced artificial intelligence for text-to-audio generation and supports the research community by providing access to pre-trained model checkpoints, which are ready for inference and available for commercial use.

Keywords: Bark, ai voice cloning, Suno, text-to-speech, artificial intelligence, audio generation, Meta's encodec, audio codebooks, semantic tokens, HuBert, transformer-based model, multilingual speech, wav2vec, linear projection head, embedding space, generative capabilities, pretrained model checkpoints
Bistatic MIMO Radar Sensing of Specularly Reflecting Surfaces for Wireless Power Transfer
Authors: Benjamin J. B. Deutschmann, Maximilian Graber, Thomas Wilding, Klaus Witrisal
Date: 2023-05-08 00:00:00

Geometric environment information aids future distributed radio infrastructures in providing services, such as ultra-reliable communication, positioning, and wireless power transfer (WPT). An a priori known environment model cannot always be assumed in practice. This paper investigates the capabilities of detecting specularly reflecting surfaces in a bistatic multiple-input multiple-output (MIMO) radar setup operating at sub-10 GHz frequencies. While rough surfaces generate diffuse reflections originating from their actual position, flat surfaces act like "mirrors," causing directive reflections that virtually originate "behind" them. Despite these propagation characteristics, we can estimate the locations of flat metal walls from reflections at their surface using synthetic aperture (SA) measurements. The performance gain achievable by exploiting this environment information is analyzed by evaluating WPT capabilities in a geometry-based beamforming setup. We show that it is possible to predict channel state information (CSI) with a geometric channel model. Our geometry-based beamformer suffers an efficiency loss of only 1.1dB compared with a reciprocity-based beamformer given perfect CSI.
Optimal Energy System Scheduling Using A Constraint-Aware Reinforcement Learning Algorithm
Authors: Hou Shengren, Pedro P. Vergara, Edgar Mauricio Salazar Duque, Peter Palensky
Date: 2023-05-09 00:00:00
Tasks: Deep Reinforcement Learning, energy management, reinforcement-learning, Reinforcement Learning (RL), Safe Reinforcement Learning, Scheduling

The massive integration of renewable-based distributed energy resources (DERs) inherently increases the energy system's complexity, especially when it comes to defining its operational schedule. Deep reinforcement learning (DRL) algorithms arise as a promising solution due to their data-driven and model-free features. However, current DRL algorithms fail to enforce rigorous operational constraints (e.g., power balance, ramping up or down constraints) limiting their implementation in real systems. To overcome this, in this paper, a DRL algorithm (namely MIP-DQN) is proposed, capable of \textit{strictly} enforcing all operational constraints in the action space, ensuring the feasibility of the defined schedule in real-time operation. This is done by leveraging recent optimization advances for deep neural networks (DNNs) that allow their representation as a MIP formulation, enabling further consideration of any action space constraints. Comprehensive numerical simulations show that the proposed algorithm outperforms existing state-of-the-art DRL algorithms, obtaining a lower error when compared with the optimal global solution (upper boundary) obtained after solving a mathematical programming formulation with perfect forecast information; while strictly enforcing all operational constraints (even in unseen test days).
ParsVQA-Caps: A Benchmark for Visual Question Answering and Image Captioning in Persian
Authors: Shaghayegh Mobasher, Ghazal Zamaninejad, Maryam Hashemi, Melika Nobakhtian, Sauleh Eetemadi
Date: 2022-12-07 00:00:00
Proceeding: WiNLP2022 2022 12
Tasks: Image Captioning, Question Answering, Visual Question Answering, Visual Question Answering (VQA)

Despite recent advances in vision-and-language tasks, most progress is still focused on resource-rich languages such as English. Furthermore, widespread vision-and-language datasets directly adopt images representative of American or European cultures resulting in bias. Hence we introduce ParsVQA-Caps, the first benchmark in Persian for Visual Question Answering and Image Captioning tasks. We utilize two ways to collect datasets for each task, human-based and template-based for VQA and human-based and web-based for image captioning.
Diffusion Kernel Attention Network for Brain Disorder Classification
Authors: Jianjia Zhang; Luping Zhou; Lei Wang; Mengting Liu; Dinggang Shen
Date: 2022-04-23 00:00:00
Proceeding: journal 2022 4
Tasks: Classification

Constructing and analyzing functional brain networks (FBN) has become a promising approach to brain disorder classification. However, the conventional successive construct-and-analyze process would limit the performance due to the lack of interactions and adaptivity among the subtasks in the process. Recently, Transformer has demonstrated remarkable performance in various tasks, attributing to its effective attention mechanism in modeling complex feature relationships. In this paper, for the first time, we develop Transformer for integrated FBN modeling, analysis and brain disorder classification with rs-fMRI data by proposing a Diffusion Kernel Attention Network to address the specific challenges. Specifically, directly applying Transformer does not necessarily admit optimal performance in this task due to its extensive parameters in the attention module against the limited training samples usually available. Looking into this issue, we propose to use kernel attention to replace the original dot-product attention module in Transformer. This significantly reduces the number of parameters to train and thus alleviates the issue of small sample while introducing a non-linear attention mechanism to model complex functional connections. Another limit of Transformer for FBN applications is that it only considers pair-wise interactions between directly connected brain regions but ignores the important indirect connections. Therefore, we further explore diffusion process over the kernel attention to incorporate wider interactions among indirectly connected brain regions. Extensive experimental study is conducted on ADHD-200 data set for ADHD classification and on ADNI data set for Alzheimer’s disease classification, and the results demonstrate the superior performance of the proposed method over the competing methods.
Low-light image enhancement based on sharpening-smoothing image filter
Authors: Y. Demir, N.H. Kaplan
Date: 2023-08-30 00:00:00
Proceeding: https://www.sciencedirect.com/science/article/pii/S1051200423001495 2023 8
Tasks: Image Enhancement, Low-Light Image Enhancement

Low-light images suffer from poor visibility, severe noise, low contrast, and low brightness. To overcome these issues, many image enhancement methods have been proposed. Few techniques solve these problems simultaneously. This paper presents a low-light image enhancement method. The proposed method first applies the HSV (Hue, Saturation, Value) transform to the input image. Here, a multi-scale decomposition of the Sharpening-Smoothing Image Filter (SSIF) is proposed to obtain approximation and detail sub-images of the V component. After the decomposition process, Contrast-Limited Adaptive Histogram Equalization (CLAHE) is applied to the final approximation image to provide higher contrast. The detail sub-images are amplified and added to the enhanced approximation image to reconstruct the enhanced V component. Finally, inverse HSV transform is applied to the enhanced V component and H, S components to obtain the enhanced image. The experimental results show that the proposed method provides better visual quality and more natural colors than the compared state-of-the-art methods.
SecurityEval Dataset: Mining Vulnerability Examples to Evaluate Machine Learning-Based Code Generation Techniques
Authors: Mohammed Latif Siddiq, Joanna C.S. Santos
Date: 2022-11-09 00:00:00
Proceeding: 1st International Workshop on Mining Software Repositories Applications for Privacy and Security (MSR4P&S ’22) 2022 11
Tasks: Code Generation

Automated source code generation is currently a popular machine-learning-based task. It can be helpful for software developers to write functionally correct code from a given context. However, just like human developers, a code generation model can produce vulnerable code, which the developers can mistakenly use. For this reason, evaluating the security of a code generation model is a must. In this paper, we describe SecurityEval, an evaluation dataset to fulfill this purpose. It contains 130 samples for 75 vulnerability types, which are mapped to the Common Weakness Enumeration (CWE). We also demonstrate using our dataset to evaluate one open-source (i.e., InCoder) and one closed-source code generation model (i.e., GitHub Copilot).