pwc-archive/papers-with-abstracts huggingface dataset:
Kaleido: You Can Watch It But Cannot Record It
Authors: Lan Zhang1, Cheng Bo2, Jiahui Hou3, 4, Xiang-Yang Li1, 3 Yu Wang2, Kebin Liu1, Yunhao Liu1 1School of Software and TNLIST, Tsinghua University, China 2Department of Computer Science, University of North Carolina at Charlotte, USA 3Department of Computer Science, Illinois Institute of Technology, USA 4Department of Computer Science, University of Science and Technology of China, China {lan, yunhao}@greenorbs.com, {cbo1, yu.wang}@uncc.edu, {jhou11, xli}@cs.iit.edu
Date: 2015-09-07 00:00:00
Proceeding: ACM 2015 9

Recently a number of systems have been developed to implement
and improve the visual communication over screen-camera links.
In this paper we study an opposite problem: how to prevent unauthorized users from videotaping a video played on a screen,
such as in a theater, while do not affect the viewing experience
of legitimate audiences. We propose and develop a light-weight
hardware-free system, called KALEIDO, that ensures these properties by taking advantage of the limited disparities between the
screen-eye channel and the screen-camera channel. KALEIDO does
not require any extra hardware and is purely based on re-encoding
the original video frame into multiple frames used for displaying.
We extensively test our system KALEIDO using a variety of smartphone cameras. Our experiments confirm that KALEIDO preserves
the high-quality screen-eye channel while reducing the secondary
screen-camera channel quality significantly.
From Components to Converters: A Fundamental Topology Derivation Method for Single-Inductor Multi-Input Multi-Output Converters Based on Graph Theory
Authors: Liping Mo, Guipeng Chen, Jiangming Huang
Date: 2021-01-01 00:00:00
Proceeding: IEEE International Conference on Power Electronics, Computer Applications (ICPECA) 2021 1

As power electronics converters can be essentially regarded as combinations of different components, a fundamental topology derivation method is proposed to derive effective topologies directly from separated components for single-inductor multi-input multi-output (SI-MIMO) converters in this paper. In the proposed components to converters (C2C) method, there are two critical puzzles to be solved. As the possible connections of components are numerous, how to conveniently and comprehensively acquire viable converter structures and identify effective topologies are difficult to be implemented through pure manual effort. And it will be further deteriorated with the increasing number of components. Therefore, graph theory is also utilized in the proposed C2C method, with which computer code can be developed to automatically complete the topology derivation. In the paper, how to employ graph theory to solve the two puzzles of C2C method is firstly introduced. Then, detailed topology derivation of three-port single-inductor converters with 3 switches and 4 switches are also shown to validate its effectiveness.
Smart Word Suggestions for Writing Assistance
Authors: Chenshuo Wang, Shaoguang Mao, Tao Ge, Wenshan Wu, Xun Wang, Yan Xia, Jonathan Tien, Dongyan Zhao
Date: 2023-05-17 00:00:00

Enhancing word usage is a desired feature for writing assistance. To further advance research in this area, this paper introduces "Smart Word Suggestions" (SWS) task and benchmark. Unlike other works, SWS emphasizes end-to-end evaluation and presents a more realistic writing assistance scenario. This task involves identifying words or phrases that require improvement and providing substitution suggestions. The benchmark includes human-labeled data for testing, a large distantly supervised dataset for training, and the framework for evaluation. The test data includes 1,000 sentences written by English learners, accompanied by over 16,000 substitution suggestions annotated by 10 native speakers. The training dataset comprises over 3.7 million sentences and 12.7 million suggestions generated through rules. Our experiments with seven baselines demonstrate that SWS is a challenging task. Based on experimental analysis, we suggest potential directions for future research on SWS. The dataset and related codes is available at https://github.com/microsoft/SmartWordSuggestions.
Tinto: Multisensor Benchmark for 3D Hyperspectral Point Cloud Segmentation in the Geosciences
Authors: Ahmed J. Afifi, Samuel T. Thiele, Aldino Rizaldy, Sandra Lorenz, Pedram Ghamisi, Raimon Tolosana-Delgado, Moritz Kirsch, Richard Gloaguen, Michael Heizmann
Date: 2023-05-17 00:00:00
Tasks: Deep Learning, Point Cloud Segmentation

The increasing use of deep learning techniques has reduced interpretation time and, ideally, reduced interpreter bias by automatically deriving geological maps from digital outcrop models. However, accurate validation of these automated mapping approaches is a significant challenge due to the subjective nature of geological mapping and the difficulty in collecting quantitative validation data. Additionally, many state-of-the-art deep learning methods are limited to 2D image data, which is insufficient for 3D digital outcrops, such as hyperclouds. To address these challenges, we present Tinto, a multi-sensor benchmark digital outcrop dataset designed to facilitate the development and validation of deep learning approaches for geological mapping, especially for non-structured 3D data like point clouds. Tinto comprises two complementary sets: 1) a real digital outcrop model from Corta Atalaya (Spain), with spectral attributes and ground-truth data, and 2) a synthetic twin that uses latent features in the original datasets to reconstruct realistic spectral data (including sensor noise and processing artifacts) from the ground-truth. The point cloud is dense and contains 3,242,964 labeled points. We used these datasets to explore the abilities of different deep learning approaches for automated geological mapping. By making Tinto publicly available, we hope to foster the development and adaptation of new deep learning tools for 3D applications in Earth sciences. The dataset can be accessed through this link: https://doi.org/10.14278/rodare.2256.
Simulate Bumblebee and Extend It to Support LE Coded PHY in BLE version 5
Authors: Hao Zhao
Date: 2023-05-17 00:00:00

Bumblebee is an innovative backscatter system that leverages productive Bluetooth Low Energy (BLE) carriers to generate ZigBee transmissions. It improves upon previous systems like FreeRider and Interscatter by eliminating restrictions on exciters and receivers, resulting in reduced deployment costs and increased throughput. To enable long-distance signal transmission, BLE version 5 introduces LE Coded PHY. However, the paper proposing Bumblebee only evaluates it with BLE version 4, which utilizes LE Uncoded PHY. In this paper, we initially simulate Bumblebee using MATLAB to assess its design. Subsequently, we extend Bumblebee to support LE Coded PHY in BLE version 5 and conduct experiments to verify its performance.
Accelerating Transformer Inference for Translation via Parallel Decoding
Authors: Andrea Santilli, Silvio Severino, Emilian Postolache, Valentino Maiorca, Michele Mancusi, Riccardo Marin, Emanuele Rodolà
Date: 2023-05-17 00:00:00
Tasks: Machine Translation, Translation

Autoregressive decoding limits the efficiency of transformers for Machine Translation (MT). The community proposed specific network architectures and learning-based methods to solve this issue, which are expensive and require changes to the MT model, trading inference speed at the cost of the translation quality. In this paper, we propose to address the problem from the point of view of decoding algorithms, as a less explored but rather compelling direction. We propose to reframe the standard greedy autoregressive decoding of MT with a parallel formulation leveraging Jacobi and Gauss-Seidel fixed-point iteration methods for fast inference. This formulation allows to speed up existing models without training or modifications while retaining translation quality. We present three parallel decoding algorithms and test them on different languages and models showing how the parallelization introduces a speedup up to 38% w.r.t. the standard autoregressive decoding and nearly 2x when scaling the method on parallel resources. Finally, we introduce a decoding dependency graph visualizer (DDGviz) that let us see how the model has learned the conditional dependence between tokens and inspect the decoding procedure.
SLiC-HF: Sequence Likelihood Calibration with Human Feedback
Authors: Yao Zhao, Rishabh Joshi, Tianqi Liu, Misha Khalman, Mohammad Saleh, Peter J. Liu
Date: 2023-05-17 00:00:00
Tasks: Language Modeling, Language Modelling, Offline RL

Learning from human feedback has been shown to be effective at aligning language models with human preferences. Past work has often relied on Reinforcement Learning from Human Feedback (RLHF), which optimizes the language model using reward scores assigned from a reward model trained on human preference data. In this work we show how the recently introduced Sequence Likelihood Calibration (SLiC), can also be used to effectively learn from human preferences (SLiC-HF). Furthermore, we demonstrate this can be done with human feedback data collected for a different model, similar to off-policy, offline RL data. Automatic and human evaluation experiments on the TL;DR summarization task show that SLiC-HF significantly improves supervised fine-tuning baselines. Furthermore, SLiC-HF presents a competitive alternative to the PPO RLHF implementation used in past work while being much simpler to implement, easier to tune and more computationally efficient in practice.
Scratch Copilot Evaluation: Assessing AI-Assisted Creative Coding for Families
Authors: Stefania Druga, Nancy Otero
Date: 2023-05-17 00:00:00

How can AI enhance creative coding experiences for families? This study explores the potential of large language models (LLMs) in helping families with creative coding using Scratch. Based on our previous user study involving a prototype AI assistant, we devised three evaluation scenarios to determine if LLMs could help families comprehend game code, debug programs, and generate new ideas for future projects. We utilized 22 Scratch projects for each scenario and generated responses from LLMs with and without practice tasks, resulting in 120 creative coding support scenario datasets. In addition, the authors independently evaluated their precision, pedagogical value, and age-appropriate language. Our findings show that LLMs achieved an overall success rate of more than 80\% on the different tasks and evaluation criteria. This research offers valuable information on using LLMs for creative family coding and presents design guidelines for future AI-supported coding applications. Our evaluation framework, together with our labeled evaluation data, is publicly available.
AI Friends: A Design Framework for AI-Powered Creative Programming for Youth
Authors: Stefania Druga, Amy J. Ko
Date: 2023-05-17 00:00:00

What role can AI play in supporting and constraining creative coding by families? To investigate these questions, we built a Wizard of Oz platform to help families engage in creative coding in partnership with a researcher-operated AI Friend. We designed a 3 week series of programming activities with ten children, 7 to 12 years old, and nine parents. Using a creative self efficacy lens, we observe that families found it easier to generate game ideas when prompted with questions by AI Friend; parents played a unique role in guiding children in more complex programming tasks when the AI Friend failed to help, and children were more encouraged to write code for novel ideas using the AI friend help. These findings suggest that AI supported platforms should highlight unique family AI interactions focused on children's agency and creative self-efficacy.
Variational Classification
Authors: Shehzaad Dhuliawala, Mrinmaya Sachan, Carl Allen
Date: 2023-05-17 00:00:00
Tasks: Adversarial Robustness, Classification, text-classification, Text Classification

We present a latent variable model for classification that provides a novel probabilistic interpretation of neural network softmax classifiers. We derive a variational objective to train the model, analogous to the evidence lower bound (ELBO) used to train variational auto-encoders, that generalises the softmax cross-entropy loss. Treating inputs to the softmax layer as samples of a latent variable, our abstracted perspective reveals a potential inconsistency between their anticipated distribution, required for accurate label predictions, and their empirical distribution found in practice. We augment the variational objective to mitigate such inconsistency and induce a chosen latent distribution, instead of the implicit assumption found in a standard softmax layer. Overall, we provide new theoretical insight into the inner workings of widely-used softmax classifiers. Empirical evaluation on image and text classification datasets demonstrates that our proposed approach, variational classification, maintains classification accuracy while the reshaped latent space improves other desirable properties of a classifier, such as calibration, adversarial robustness, robustness to distribution shift and sample efficiency useful in low data settings.
Explainable Multi-Agent Reinforcement Learning for Temporal Queries
Authors: Kayla Boggess, Sarit Kraus, Lu Feng
Date: 2023-05-17 00:00:00
Tasks: Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning

As multi-agent reinforcement learning (MARL) systems are increasingly deployed throughout society, it is imperative yet challenging for users to understand the emergent behaviors of MARL agents in complex environments. This work presents an approach for generating policy-level contrastive explanations for MARL to answer a temporal user query, which specifies a sequence of tasks completed by agents with possible cooperation. The proposed approach encodes the temporal query as a PCTL logic formula and checks if the query is feasible under a given MARL policy via probabilistic model checking. Such explanations can help reconcile discrepancies between the actual and anticipated multi-agent behaviors. The proposed approach also generates correct and complete explanations to pinpoint reasons that make a user query infeasible. We have successfully applied the proposed approach to four benchmark MARL domains (up to 9 agents in one domain). Moreover, the results of a user study show that the generated explanations significantly improve user performance and satisfaction.
Human Choice Prediction in Language-based Persuasion Games: Simulation-based Off-Policy Evaluation
Authors: Eilam Shapira, Omer Madmon, Reut Apel, Moshe Tennenholtz, Roi Reichart
Date: 2023-05-17 00:00:00
Tasks: Decision Making, Off-policy evaluation

Recent advances in Large Language Models (LLMs) have spurred interest in designing LLM-based agents for tasks that involve interaction with human and artificial agents. This paper addresses a key aspect in the design of such agents: predicting human decisions in off-policy evaluation (OPE). We focus on language-based persuasion games, where an expert aims to influence the decision-maker through verbal messages. In our OPE framework, the prediction model is trained on human interaction data collected from encounters with one set of expert agents, and its performance is evaluated on interactions with a different set of experts. Using a dedicated application, we collected a dataset of 87K decisions from humans playing a repeated decision-making game with artificial agents. To enhance off-policy performance, we propose a simulation technique involving interactions across the entire agent space and simulated decision-makers. Our learning strategy yields significant OPE gains, e.g., improving prediction accuracy in the top 15% challenging cases by 7.1%. Our code and the large dataset we collected and generated are submitted as supplementary material and publicly available in our GitHub repository: https://github.com/eilamshapira/HumanChoicePrediction
Interactive Learning of Hierarchical Tasks from Dialog with GPT
Authors: Lane Lawley, Christopher J. MacLellan
Date: 2023-05-17 00:00:00

We present a system for interpretable, symbolic, interactive task learning from dialog using a GPT model as a conversational front-end. The learned tasks are represented as hierarchical decompositions of predicate-argument structures with scoped variable arguments. By using a GPT model to convert interactive dialog into a semantic representation, and then recursively asking for definitions of unknown steps, we show that hierarchical task knowledge can be acquired and re-used in a natural and unrestrained conversational environment. We compare our system to a similar architecture using a more conventional parser and show that our system tolerates a much wider variety of linguistic variance.
Automatic Photo Orientation Detection with Convolutional Neural Networks
Authors: Ujash Joshi, Michael Guerzhoy
Date: 2023-05-17 00:00:00

We apply convolutional neural networks (CNN) to the problem of image orientation detection in the context of determining the correct orientation (from 0, 90, 180, and 270 degrees) of a consumer photo. The problem is especially important for digitazing analog photographs. We substantially improve on the published state of the art in terms of the performance on one of the standard datasets, and test our system on a more difficult large dataset of consumer photos. We use Guided Backpropagation to obtain insights into how our CNN detects photo orientation, and to explain its mistakes.
LeTI: Learning to Generate from Textual Interactions
Authors: Xingyao Wang, Hao Peng, Reyhaneh Jabbarvand, Heng Ji
Date: 2023-05-17 00:00:00
Tasks: Code Generation, Event Argument Extraction, HumanEval, mbpp

Fine-tuning pre-trained language models (LMs) is essential for enhancing their capabilities. Existing techniques commonly fine-tune on input-output pairs (e.g., instruction tuning) or with numerical rewards that gauge the output quality (e.g., RLHF). We explore LMs' potential to learn from textual interactions (LETI) that not only check their correctness with binary labels but also pinpoint and explain errors in their outputs through textual feedback. Our focus is the code generation task, where the model produces code based on natural language instructions. This setting invites a natural and scalable way to acquire textual feedback: the error messages and stack traces from code execution using a Python interpreter. LETI iteratively fine-tunes the model, using the LM objective, on a concatenation of natural language instructions, LM-generated programs, and textual feedback. Prepended to this fine-tuning text, a binary reward token is used to differentiate correct and buggy solutions. LETI requires no ground-truth outputs for training and even outperforms a fine-tuned baseline that does. LETI not only improves the performance of LMs on a code generation dataset MBPP, but also generalizes to other datasets. Trained on MBPP, it achieves comparable or better performance than the base LMs on unseen problems in HumanEval. Furthermore, compared to binary feedback, we observe that textual feedback leads to improved generation quality and sample efficiency, achieving the same performance with fewer than half of the gradient steps. LETI is equally applicable in natural language tasks when they can be formulated as code generation, which we empirically verified on event argument extraction.
Rethinking Data Augmentation for Tabular Data in Deep Learning
Authors: Soma Onishi, Shoya Meguro
Date: 2023-05-17 00:00:00
Tasks: Contrastive Learning, Data Augmentation, Deep Learning, Self-Supervised Learning

Tabular data is the most widely used data format in machine learning (ML). While tree-based methods outperform DL-based methods in supervised learning, recent literature reports that self-supervised learning with Transformer-based models outperforms tree-based methods. In the existing literature on self-supervised learning for tabular data, contrastive learning is the predominant method. In contrastive learning, data augmentation is important to generate different views. However, data augmentation for tabular data has been difficult due to the unique structure and high complexity of tabular data. In addition, three main components are proposed together in existing methods: model structure, self-supervised learning methods, and data augmentation. Therefore, previous works have compared the performance without comprehensively considering these components, and it is not clear how each component affects the actual performance. In this study, we focus on data augmentation to address these issues. We propose a novel data augmentation method, $\textbf{M}$ask $\textbf{T}$oken $\textbf{R}$eplacement ($\texttt{MTR}$), which replaces the mask token with a portion of each tokenized column; $\texttt{MTR}$ takes advantage of the properties of Transformer, which is becoming the predominant DL-based architecture for tabular data, to perform data augmentation for each column embedding. Through experiments with 13 diverse public datasets in both supervised and self-supervised learning scenarios, we show that $\texttt{MTR}$ achieves competitive performance against existing data augmentation methods and improves model performance. In addition, we discuss specific scenarios in which $\texttt{MTR}$ is most effective and identify the scope of its application. The code is available at https://github.com/somaonishi/MTR/.
UniEX: An Effective and Efficient Framework for Unified Information Extraction via a Span-extractive Perspective
Authors: Ping Yang, Junyu Lu, Ruyi Gan, Junjie Wang, Yuxiang Zhang, Jiaxing Zhang, Pingjian Zhang
Date: 2023-05-17 00:00:00
Tasks: Event Extraction, named-entity-recognition, Named Entity Recognition, Relation Extraction, Sentiment Analysis

We propose a new paradigm for universal information extraction (IE) that is compatible with any schema format and applicable to a list of IE tasks, such as named entity recognition, relation extraction, event extraction and sentiment analysis. Our approach converts the text-based IE tasks as the token-pair problem, which uniformly disassembles all extraction targets into joint span detection, classification and association problems with a unified extractive framework, namely UniEX. UniEX can synchronously encode schema-based prompt and textual information, and collaboratively learn the generalized knowledge from pre-defined information using the auto-encoder language models. We develop a traffine attention mechanism to integrate heterogeneous factors including tasks, labels and inside tokens, and obtain the extraction target via a scoring matrix. Experiment results show that UniEX can outperform generative universal IE models in terms of performance and inference-speed on $14$ benchmarks IE datasets with the supervised setting. The state-of-the-art performance in low-resource scenarios also verifies the transferability and effectiveness of UniEX.