pwc-archive/papers-with-abstracts huggingface dataset:
Inducing High Energy-Latency of Large Vision-Language Models with Verbose Images
Authors: Kuofeng Gao, Yang Bai, Jindong Gu, Shu-Tao Xia, Philip Torr, Zhifeng Li, Wei Liu
Date: 2024-01-20 00:00:00
Tasks: Diversity

Large vision-language models (VLMs) such as GPT-4 have achieved exceptional performance across various multi-modal tasks. However, the deployment of VLMs necessitates substantial energy consumption and computational resources. Once attackers maliciously induce high energy consumption and latency time (energy-latency cost) during inference of VLMs, it will exhaust computational resources. In this paper, we explore this attack surface about availability of VLMs and aim to induce high energy-latency cost during inference of VLMs. We find that high energy-latency cost during inference of VLMs can be manipulated by maximizing the length of generated sequences. To this end, we propose verbose images, with the goal of crafting an imperceptible perturbation to induce VLMs to generate long sentences during inference. Concretely, we design three loss objectives. First, a loss is proposed to delay the occurrence of end-of-sequence (EOS) token, where EOS token is a signal for VLMs to stop generating further tokens. Moreover, an uncertainty loss and a token diversity loss are proposed to increase the uncertainty over each generated token and the diversity among all tokens of the whole generated sequence, respectively, which can break output dependency at token-level and sequence-level. Furthermore, a temporal weight adjustment algorithm is proposed, which can effectively balance these losses. Extensive experiments demonstrate that our verbose images can increase the length of generated sequences by 7.87 times and 8.56 times compared to original images on MS-COCO and ImageNet datasets, which presents potential challenges for various applications. Our code is available at https://github.com/KuofengGao/Verbose_Images.
Forecasting Cryptocurrency Staking Rewards
Authors: Sauren Gupta, Apoorva Hathi Katharaki, Yifan Xu, Bhaskar Krishnamachari, Rajarshi Gupta
Date: 2024-01-16 00:00:00
Tasks: regression

This research explores a relatively unexplored area of predicting cryptocurrency staking rewards, offering potential insights to researchers and investors. We investigate two predictive methodologies: a) a straightforward sliding-window average, and b) linear regression models predicated on historical data. The findings reveal that ETH staking rewards can be forecasted with an RMSE within 0.7% and 1.1% of the mean value for 1-day and 7-day look-aheads respectively, using a 7-day sliding-window average approach. Additionally, we discern diverse prediction accuracies across various cryptocurrencies, including SOL, XTZ, ATOM, and MATIC. Linear regression is identified as superior to the moving-window average for perdicting in the short term for XTZ and ATOM. The results underscore the generally stable and predictable nature of staking rewards for most assets, with MATIC presenting a noteworthy exception.
A Finger on the Pulse of Cardiovascular Health: Estimating Blood Pressure with Smartphone Photoplethysmography-Based Pulse Waveform Analysis
Authors: Ivan Liu, Fangyuan Liu, Qi Zhong, Shiguang Ni
Date: 2024-01-20 00:00:00

Utilizing mobile phone cameras for continuous blood pressure (BP) monitoring presents a cost-effective and accessible approach, yet it is challenged by limitations in accuracy and interpretability. This study introduces four innovative strategies to enhance smartphone-based photoplethysmography for BP estimation (SPW-BP), addressing the interpretability-accuracy dilemma. First, we employ often-neglected data-quality improvement techniques, such as height normalization, corrupt data removal, and boundary signal reconstruction. Second, we conduct a comprehensive analysis of thirty waveform indicators across three categories to identify the most predictive features. Third, we use SHapley Additive exPlanations (SHAP) analysis to ensure the transparency and explainability of machine learning outcomes. Fourth, we utilize Bland-Altman analysis alongside AAMI and BHS standards for comparative evaluation. Data from 127 participants demonstrated a significant correlation between smartphone-captured waveform features and those from standard BP monitoring devices. Employing multiple linear regression within a cross-validation framework, waveform variables predicted systolic blood pressure (SBP) with a mean absolute error (MAE) of 3.08-16.64 mmHg and diastolic blood pressure (DBP) with an MAE of 2.86-13.16 mmHg. Further application of Random Forest models significantly improved the prediction MAE for SBP to 2.61-15.21 mmHg and for DBP to 2.14-11.22 mmHg, indicating enhanced predictive accuracy. Correlation and SHAP analysis identified key features for improving BP estimation. However, Bland-Altman analysis revealed systematic biases, and MAE analysis showed that the results did not meet AAMI and BHS accuracy standards. Our findings highlight the potential of SPW-BP, yet suggest that smartphone PPG technology is not yet a viable alternative to traditional medical devices for BP measurement.
Local Diversity of Condorcet Domains
Authors: Alexander Karpov, Klas Markström, Søren Riis, Bei Zhou
Date: 2024-01-22 00:00:00
Tasks: Diversity

Several of the classical results in social choice theory demonstrate that in order for many voting systems to be well-behaved the set domain of individual preferences must satisfy some kind of restriction, such as being single-peaked on a political axis. As a consequence it becomes interesting to measure how diverse the preferences in a well-behaved domain can be. In this paper we introduce an egalitarian approach to measuring preference diversity, focusing on the abundance of distinct suborders one subsets of the alternative. We provide a common generalisation of the frequently used concepts of ampleness and copiousness. We give a detailed investigation of the abundance for Condorcet domains. Our theorems imply a ceiling for the local diversity in domains on large sets of alternatives, which show that in this measure Black's single-peaked domain is in fact optimal. We also demonstrate that for some numbers of alternatives, there are Condorcet domains which have largest local diversity without having maximum order.
ColorVideoVDP: A visual difference predictor for image, video and display distortions
Authors: Rafal K. Mantiuk, Param Hanji, Maliha Ashraf, Yuta Asano, ALEXANDRE CHAPIRO
Date: 2024-01-21 00:00:00
Tasks: Video Compression

ColorVideoVDP is a video and image quality metric that models spatial and temporal aspects of vision, for both luminance and color. The metric is built on novel psychophysical models of chromatic spatiotemporal contrast sensitivity and cross-channel contrast masking. It accounts for the viewing conditions, geometric, and photometric characteristics of the display. It was trained to predict common video streaming distortions (e.g. video compression, rescaling, and transmission errors), and also 8 new distortion types related to AR/VR displays (e.g. light source and waveguide non-uniformities). To address the latter application, we collected our novel XR-Display-Artifact-Video quality dataset (XR-DAVID), comprised of 336 distorted videos. Extensive testing on XR-DAVID, as well as several datasets from the literature, indicate a significant gain in prediction performance compared to existing metrics. ColorVideoVDP opens the doors to many novel applications which require the joint automated spatiotemporal assessment of luminance and color distortions, including video streaming, display specification and design, visual comparison of results, and perceptually-guided quality optimization.
A VR Serious Game to Increase Empathy towards Students with Phonological Dyslexia
Authors: José M. Alcalde-Llergo, Enrique Yeguas-Bolívar, Pilar Aparicio-Martínez, Andrea Zingoni, Juri Taborri, Sara Pinzi
Date: 2024-01-15 00:00:00

Dyslexia is a neurodevelopmental disorder that is estimated to affect about 5-10% of the population. In particular, phonological dyslexia causes problems in connecting the sounds of words with their written forms. This results in difficulties such as slow reading speed, inaccurate reading, and difficulty decoding unfamiliar words. Moreover, dyslexia can also be a challenging and frustrating experience for students as they may feel misunderstood or stigmatized by their peers or educators. For these reasons, the use of compensatory tools and strategies is of crucial importance for dyslexic students to have the same opportunities as non-dyslexic ones. However, generally, people underestimate the problem and are not aware of the importance of support methodologies. In the light of this, the main purpose of this paper is to propose a virtual reality (VR) serious game through which teachers, students and, in general, non-dyslexic people could understand which are some of the issues of student with dyslexia and the fundamental utility of offering support to them. In the game, players must create a potion by following a recipe written in an alphabet that is specifically designed to replicate the reading difficulties experienced by individuals with dyslexia. The task must be solved first without any help and then by receiving supporting tools and strategies with the idea that the player can put himself in the place of the dyslexic person and understand the real need for support methodologies.
The Market Consequences of Perceived Strategic Generosity: An Empirical Examination of NFT Charity Fundraisers
Authors: Chen Liang, Murat Tunc, Gordon Burtch
Date: 2024-01-22 00:00:00

Crypto donations now represent a significant fraction of charitable giving worldwide. Nonfungible token (NFT) charity fundraisers, which involve the sale of NFTs of artistic works with the proceeds donated to philanthropic causes, have emerged as a novel development in this space. A unique aspect of NFT charity fundraisers is the significant potential for donors to reap financial gains from the rising value of purchased NFTs. Questions may arise about donors' motivations in these charity fundraisers, potentially resulting in a negative social image. NFT charity fundraisers thus offer a unique opportunity to understand the economic consequences of a donor's social image. We investigate these effects in the context of a large NFT charity fundraiser. We identify the causal effect of purchasing an NFT within the charity fundraiser on a donor's later market outcomes by leveraging random variation in transaction processing times on the blockchain. Further, we demonstrate a clear pattern of heterogeneity based on an individual's decision to relist (versus hold) the purchased charity NFTs (a sign of perceived strategic generosity) and based on an individual's social exposure within the NFT marketplace. We show that charity-NFT 're-listers' experience significant penalties in the market regarding the prices they can command for their other NFTs, particularly among those who are more socially exposed. Finally, we report the results of a scenario-based online experiment, which again support our findings, highlighting that the re-listing a charity NFT for sale at a profit leads others to perceive their initial donation as strategic generosity and reduces those others' willingness to purchase NFTs from the donor. Our study underscores the growing importance of digital visibility and traceability, features that characterize crypto-philanthropy, and online philanthropy more broadly.
Crowd-PrefRL: Preference-Based Reward Learning from Crowds
Authors: David Chhan, Ellen Novoseller, Vernon J. Lawhern
Date: 2024-01-17 00:00:00
Tasks: Reinforcement Learning (RL)

Preference-based reinforcement learning (RL) provides a framework to train AI agents using human feedback through preferences over pairs of behaviors, enabling agents to learn desired behaviors when it is difficult to specify a numerical reward function. While this paradigm leverages human feedback, it typically treats the feedback as given by a single human user. However, different users may desire multiple AI behaviors and modes of interaction. Meanwhile, incorporating preference feedback from crowds (i.e. ensembles of users) in a robust manner remains a challenge, and the problem of training RL agents using feedback from multiple human users remains understudied. In this work, we introduce a conceptual framework, Crowd-PrefRL, that integrates preference-based RL approaches with techniques from unsupervised crowdsourcing to enable training of autonomous system behaviors from crowdsourced feedback. We show preliminary results suggesting that Crowd-PrefRL can learn reward functions and agent policies from preference feedback provided by crowds of unknown expertise and reliability. We also show that in most cases, agents trained with Crowd-PrefRL outperform agents trained with majority-vote preferences or preferences from any individual user, especially when the spread of user error rates among the crowd is large. Results further suggest that our method can identify the presence of minority viewpoints within the crowd in an unsupervised manner.
Push- and Pull-based Effective Communication in Cyber-Physical Systems
Authors: Pietro Talli, Federico Mason, Federico Chiariotti, Andrea Zanella
Date: 2024-01-15 00:00:00

In Cyber Physical Systems (CPSs), two groups of actors interact toward the maximization of system performance: the sensors, observing and disseminating the system state, and the actuators, performing physical decisions based on the received information. While it is generally assumed that sensors periodically transmit updates, returning the feedback signal only when necessary, and consequently adapting the physical decisions to the communication policy, can significantly improve the efficiency of the system. In particular, the choice between push-based communication, in which updates are initiated autonomously by the sensors, and pull-based communication, in which they are requested by the actuators, is a key design step. In this work, we propose an analytical model for optimizing push- and pull-based communication in CPSs, observing that the policy optimality coincides with Value of Information (VoI) maximization. Our results also highlight that, despite providing a better optimal solution, implementable push-based communication strategies may underperform even in relatively simple scenarios.
Energy Consumption Analysis for Continuous Phase Modulation in Smart-Grid Internet of Things of beyond 5G
Authors: Hongjian Gao, Yang Lu, Shaoshi Yang, Jingsheng Tan, Longlong Nie, Xinyi Qu
Date: 2024-01-21 00:00:00

Wireless sensor network (WSN) underpinning the smart-grid Internet of Things (SG-IoT) has been a popular research topic in recent years due to its great potential for enabling a wide range of important applications. However, the energy consumption (EC) characteristic of sensor nodes is a key factor that affects the operational performance (e.g., lifetime of sensors) and the total cost of ownership of WSNs. In this paper, to find the modulation techniques suitable for WSNs, we investigate the EC characteristic of continuous phase modulation (CPM), which is an attractive modulation scheme candidate for WSNs because of its constant envelope property. We first develop an EC model for the sensor nodes of WSNs by considering the circuits and a typical communication protocol that relies on automatic repeat request (ARQ)-based retransmissions to ensure successful data delivery. Then, we use this model to analyze the EC characteristic of CPM under various configurations of modulation parameters. Furthermore, we compare the EC characteristic of CPM with that of other representative modulation schemes, such as offset quadrature phase-shift keying (OQPSK) and quadrature amplitude modulation (QAM), which are commonly used in communication protocols of WSNs. Our analysis and simulation results provide insights into the EC characteristics of multiple modulation schemes in the context of WSNs; thus, they are beneficial for designing energy-efficient SG-IoT in the beyond-5G (B5G) and the 6G era.
Friends Across Time: Multi-Scale Action Segmentation Transformer for Surgical Phase Recognition
Authors: Bokai Zhang, Jiayuan Meng, Bin Cheng, Dean Biskup, Svetlana Petculescu, Angela Chapman
Date: 2024-01-22 00:00:00
Tasks: Action Segmentation, Offline surgical phase recognition, Online surgical phase recognition, Segmentation, Surgical phase recognition

Automatic surgical phase recognition is a core technology for modern operating rooms and online surgical video assessment platforms. Current state-of-the-art methods use both spatial and temporal information to tackle the surgical phase recognition task. Building on this idea, we propose the Multi-Scale Action Segmentation Transformer (MS-AST) for offline surgical phase recognition and the Multi-Scale Action Segmentation Causal Transformer (MS-ASCT) for online surgical phase recognition. We use ResNet50 or EfficientNetV2-M for spatial feature extraction. Our MS-AST and MS-ASCT can model temporal information at different scales with multi-scale temporal self-attention and multi-scale temporal cross-attention, which enhances the capture of temporal relationships between frames and segments. We demonstrate that our method can achieve 95.26% and 96.15% accuracy on the Cholec80 dataset for online and offline surgical phase recognition, respectively, which achieves new state-of-the-art results. Our method can also achieve state-of-the-art results on non-medical datasets in the video action segmentation domain.
EndoGS: Deformable Endoscopic Tissues Reconstruction with Gaussian Splatting
Authors: Lingting Zhu, Zhao Wang, Jiahao Cui, Zhenchao Jin, Guying Lin, Lequan Yu
Date: 2024-01-21 00:00:00
Tasks: 3D Reconstruction

Surgical 3D reconstruction is a critical area of research in robotic surgery, with recent works adopting variants of dynamic radiance fields to achieve success in 3D reconstruction of deformable tissues from single-viewpoint videos. However, these methods often suffer from time-consuming optimization or inferior quality, limiting their adoption in downstream tasks. Inspired by 3D Gaussian Splatting, a recent trending 3D representation, we present EndoGS, applying Gaussian Splatting for deformable endoscopic tissue reconstruction. Specifically, our approach incorporates deformation fields to handle dynamic scenes, depth-guided supervision with spatial-temporal weight masks to optimize 3D targets with tool occlusion from a single viewpoint, and surface-aligned regularization terms to capture the much better geometry. As a result, EndoGS reconstructs and renders high-quality deformable endoscopic tissues from a single-viewpoint video, estimated depth maps, and labeled tool masks. Experiments on DaVinci robotic surgery videos demonstrate that EndoGS achieves superior rendering quality. Code is available at https://github.com/HKU-MedAI/EndoGS.
Consistency Based Unsupervised Self-training For ASR Personalisation
Authors: Jisi Zhang, Vandana Rajan, Haaris Mehmood, David Tuckey, Pablo Peso Parada, Md Asif Jalal, Karthikeyan Saravanan, Gil Ho Lee, Jungin Lee, Seokyeong Jung
Date: 2024-01-22 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

On-device Automatic Speech Recognition (ASR) models trained on speech data of a large population might underperform for individuals unseen during training. This is due to a domain shift between user data and the original training data, differed by user's speaking characteristics and environmental acoustic conditions. ASR personalisation is a solution that aims to exploit user data to improve model robustness. The majority of ASR personalisation methods assume labelled user data for supervision. Personalisation without any labelled data is challenging due to limited data size and poor quality of recorded audio samples. This work addresses unsupervised personalisation by developing a novel consistency based training method via pseudo-labelling. Our method achieves a relative Word Error Rate Reduction (WERR) of 17.3% on unlabelled training data and 8.1% on held-out data compared to a pre-trained model, and outperforms the current state-of-the art methods.
Adversarial speech for voice privacy protection from Personalized Speech generation
Authors: Shihao Chen, Liping Chen, Jie Zhang, KongAik Lee, ZhenHua Ling, LiRong Dai
Date: 2024-01-22 00:00:00
Tasks: Speaker Verification, text-to-speech, Text to Speech, Voice Conversion

The rapid progress in personalized speech generation technology, including personalized text-to-speech (TTS) and voice conversion (VC), poses a challenge in distinguishing between generated and real speech for human listeners, resulting in an urgent demand in protecting speakers' voices from malicious misuse. In this regard, we propose a speaker protection method based on adversarial attacks. The proposed method perturbs speech signals by minimally altering the original speech while rendering downstream speech generation models unable to accurately generate the voice of the target speaker. For validation, we employ the open-source pre-trained YourTTS model for speech generation and protect the target speaker's speech in the white-box scenario. Automatic speaker verification (ASV) evaluations were carried out on the generated speech as the assessment of the voice protection capability. Our experimental results show that we successfully perturbed the speaker encoder of the YourTTS model using the gradient-based I-FGSM adversarial perturbation method. Furthermore, the adversarial perturbation is effective in preventing the YourTTS model from generating the speech of the target speaker. Audio samples can be found in https://voiceprivacy.github.io/Adeversarial-Speech-with-YourTTS.
Acoustic Disturbance Sensing Level Detection for ASD Diagnosis and Intelligibility Enhancement
Authors: Marcelo Pillonetto, Anderson Queiroz, Rosângela Coelho
Date: 2024-01-22 00:00:00

The acoustic sensitivity of Autism Spectrum Disorder (ASD) individuals highly impacts their intelligibility in noisy urban environments. In this Letter, the disturbance sensing level is examined with perceptual listening tests that demonstrate the impact of their append High Internal Noise (HIN) profile on intelligibility. This particular sensing level is then proposed as additional aid to ASD diagnosis. In this Letter, a novel intelligibility enhancement scheme is also introduced for ASD particular circumstances. For this proposal, harmonic features estimated from speech signal frames are considered as center frequencies of auditory filterbanks. A gain factor is further applied to the output of the filtered samples. The experimental results demonstrate that the proposal improved the acoustic intelligibility of ASD and Neurotypicals (NT) people considering four acoustic noises at different signal-to-noise ratios.