pwc-archive/papers-with-abstracts huggingface dataset:
You Don't Know Search: Helping Users Find Code by Automatically Evaluating Alternative Queries
Authors: Rijnard van Tonder
Date: 2022-12-07 00:00:00
Tasks: Code Search

Tens of thousands of engineers use Sourcegraph day-to-day to search for code and rely on it to make progress on software development tasks. We face a key challenge in designing a query language that accommodates the needs of a broad spectrum of users. Our experience shows that users express different and often contradictory preferences for how queries should be interpreted. These preferences stem from users with differing usage contexts, technical experience, and implicit expectations from using prior tools. At the same time, designing a code search query language poses unique challenges because it intersects traditional search engines and full-fledged programming languages. For example, code search queries adopt certain syntactic conventions in the interest of simplicity and terseness but invariably risk encoding implicit semantics that are ambiguous at face-value (a single space in a query could mean three or more semantically different things depending on surrounding terms). Users often need to disambiguate intent with additional syntax so that a query expresses what they actually want to search. This need to disambiguate is one of the primary frustrations we've seen users experience with writing search queries in the last three years. We share our observations that lead us to a fresh perspective where code search behavior can straddle seemingly ambiguous queries. We develop Automated Query Evaluation (AQE), a new technique that automatically generates and adaptively runs alternative query interpretations in frustration-prone conditions. We evaluate AQE with an A/B test across more than 10,000 unique users on our publicly-available code search instance. Our main result shows that relative to the control group, users are on average 22% more likely to click on a search result at all on any given day when AQE is active.
Improved Speech Pre-Training with Supervision-Enhanced Acoustic Unit
Authors: Pengcheng Li, Genshun Wan, Fenglin Ding, Hang Chen, Jianqing Gao, Jia Pan, Cong Liu
Date: 2022-12-07 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Self-Supervised Learning, speech-recognition, Speech Recognition

Speech pre-training has shown great success in learning useful and general latent representations from large-scale unlabeled data. Based on a well-designed self-supervised learning pattern, pre-trained models can be used to serve lots of downstream speech tasks such as automatic speech recognition. In order to take full advantage of the labed data in low resource task, we present an improved pre-training method by introducing a supervision-enhanced acoustic unit (SEAU) pattern to intensify the expression of comtext information and ruduce the training cost. Encoder representations extracted from the SEAU pattern are used to generate more representative target units for HuBERT pre-training process. The proposed method, named SeHuBERT, achieves a relative word error rate reductions of 10.5% and 4.9% comared with the standard HuBERT on Turkmen speech recognition task with 500 hours and 100 hours fine-tuning data respectively. Extended to more languages and more data, SeHuBERT can aslo achieve a relative word error rate reductions of approximately 10% at half of the training cost compared with HuBERT.
GaitVibe+: Enhancing Structural Vibration-based Footstep Localization Using Temporary Cameras for In-home Gait Analysis
Authors: Yiwen Dong, Jingxiao Liu, Hae Young Noh
Date: 2022-12-07 00:00:00
Tasks: Event Extraction, parameter estimation

In-home gait analysis is important for providing early diagnosis and adaptive treatments for individuals with gait disorders. Existing systems include wearables and pressure mats, but they have limited scalability. Recent studies have developed vision-based systems to enable scalable, accurate in-home gait analysis, but it faces privacy concerns due to the exposure of people's appearances. Our prior work developed footstep-induced structural vibration sensing for gait monitoring, which is device-free, wide-ranged, and perceived as more privacy-friendly. Although it has succeeded in temporal gait event extraction, it shows limited performance for spatial gait parameter estimation due to imprecise footstep localization. In particular, the localization error mainly comes from the estimation error of the wave arrival time at the vibration sensors and its error propagation to wave velocity estimations. Therefore, we present GaitVibe+, a vibration-based footstep localization method fused with temporarily installed cameras for in-home gait analysis. Our method has two stages: fusion and operating. In the fusion stage, both cameras and vibration sensors are installed to record only a few trials of the subject's footstep data, through which we characterize the uncertainty in wave arrival time and model the wave velocity profiles for the given structure. In the operating stage, we remove the camera to preserve privacy at home. The footstep localization is conducted by estimating the time difference of arrival (TDoA) over multiple vibration sensors, whose accuracy is improved through the reduced uncertainty and velocity modeling during the fusion stage. We evaluate GaitVibe+ through a real-world experiment with 50 walking trials. With only 3 trials of multi-modal fusion, our approach has an average localization error of 0.22 meters, which reduces the spatial gait parameter error from 111% to 27%.
Editing a Woman's Voice
Authors: Anna Costello, Ekaterina Fedorova, Zhijing Jin, Rada Mihalcea
Date: 2022-12-05 00:00:00

Prior work shows that men and women speak with different levels of confidence, though it's often assumed that these differences are innate or are learned in early childhood. Using academic publishing as a setting, we find that language differences across male and female authors are initially negligible: in first drafts of academic manuscripts, men and women write with similar levels of uncertainty. However, when we trace those early drafts to their published versions, a substantial gender gap in linguistic uncertainty arises. That is, women increase their use of cautionary language through the publication process more than men. We show this increase in the linguistic gender gap varies substantially based on editor assignment. Specifically, our author-to-editor matched dataset allows us to estimate editor-specific fixed effects, capturing how specific editors impact the change in linguistic uncertainty for female authors relative to male authors (the editor's author-gender gap). Editors' author-gender gaps vary widely, and correlate with observable editor characteristics such as societal norms in their country-of-origin, their work history, and the year that they obtained their PhD. Overall, our study suggests that a woman's "voice" is partially shaped by external forces, and it highlights the critical role of editors in shaping how female academics communicate.
Oppressed species can form a winning pair in a multi-species ecosystem
Authors: Attila Szolnoki, Matjaz Perc
Date: 2022-12-06 00:00:00
Tasks: Form, MORPH

The self-protection of alliances against external invaders is a key concept behind the maintenance of biodiversity in the face of natural selection. But since these alliances, which can be formed by different numbers of competitors, can also compete against each other, it is important to identify their strengths and weaknesses. Here, we therefore compare the vitalities of two two-species alliances whose members either beat each other mutually via a bidirectional invasion or they exchange their positions during an inner dynamics. The resulting four-species model shows rich behavior in dependence on the model parameter $p$, which characterizes the inner invasions, and $\beta$, which determines the intensity of site exchanges. In the low $p$ and the large $p$ limit, when the inner invasion becomes biased, three-member rock-scissors-paper-type solutions emerge, where one of the members is oppressed by having the smallest average concentration due to heterogeneous inner invasion rates. Interestingly, however, if we allow a more intensive site exchange between the oppressed species, they can morph into a winning pair and dominate the full parameter plane. We show that their victory utilizes the vulnerability of the rival alliance based on cyclic dominance, where a species can easily fixate a limited-size domain.
BiPMAP: A Toolbox for Predictions of Perceived Motion Artifacts on Modern Displays
Authors: Guanghan Meng, Dekel Galor, Laura Waller, Martin S. Banks
Date: 2022-12-07 00:00:00
Tasks: GPU

Presenting dynamic scenes without incurring motion artifacts visible to observers requires sustained effort from the display industry. A tool that predicts motion artifacts and simulates artifact elimination through optimizing the display configuration is highly desired to guide the design and manufacture of modern displays. Despite the popular demands, there is no such tool available in the market. In this study, we deliver an interactive toolkit, Binocular Perceived Motion Artifact Predictor (BiPMAP), as an executable file with GPU acceleration. BiPMAP accounts for an extensive collection of user-defined parameters and directly visualizes a variety of motion artifacts by presenting the perceived continuous and sampled moving stimuli side-by-side. For accurate artifact predictions, BiPMAP utilizes a novel model of the human contrast sensitivity function to effectively imitate the frequency modulation of the human visual system. In addition, BiPMAP is capable of deriving various in-plane motion artifacts for 2D displays and depth distortion in 3D stereoscopic displays.
End-to-end Optimization of Constellation Shaping for Wiener Phase Noise Channels with a Differentiable Blind Phase Search
Authors: Andrej Rode, Benedikt Geiger, Shrinivas Chimmalgi, Laurent Schmalen
Date: 2022-12-07 00:00:00

As the demand for higher data throughput in coherent optical communication systems increases, we need to find ways to increase capacity in existing and future optical communication links. To address the demand for higher spectral efficiencies, we apply end-to-end optimization for joint geometric and probabilistic constellation shaping in the presence of Wiener phase noise and carrier phase estimation. Our approach follows state-of-the-art bitwise auto-encoders, which require a differentiable implementation of all operations between transmitter and receiver, including the DSP algorithms. In this work, we show how to modify the ubiquitous blind phase search (BPS) algorithm, a popular carrier phase estimation algorithm, to make it differentiable and include it in the end-to-end constellation shaping. By leveraging joint geometric and probabilistic constellation shaping, we are able to obtain a robust and pilot-free modulation scheme improving the performance of 64-ary communication systems by at least 0.1bit/symbol compared to square QAM constellations with neural demappers and by 0.05 bit/symbol compared to previously presented approaches applying only geometric constellation shaping.
A Novel Analytical Model for LEO and MEO Satellite Networks based on Cox Point Processes
Authors: Chang-Sik Choi, François Baccelli
Date: 2022-12-07 00:00:00
Tasks: Point Processes

This work develops an analytical framework for downlink low Earth orbit (LEO) or medium Earth orbit (MEO) satellite communications, leveraging tools from stochastic geometry. We propose a tractable approach to the analysis of such satellite communication systems, accounting for the fact that satellites are located on circular orbits. We accurately incorporate this geometric property of LEO or MEO satellite constellations by developing a Cox point process model that jointly produces orbits and satellites on these orbits. Our work contrasts with previous modeling studies that presumed satellite locations to be entirely random, thereby overlooking the fundamental fact that satellites are jointly positioned on orbits. Employing this Cox model, we analyze the network performance experienced by users located on Earth. Specifically, we evaluate the no-satellite probability of the proposed network and the Laplace transform of the interference created by such a network. Using it, we compute its SIR (signal-to-interference) distribution, namely its coverage probability. By presenting fundamental network performance as functions of key parameters, this model allows one to assess the statistical properties of downlink LEO or MEO satellite communications and can thus be used as a system-level design tool to operate and optimize forthcoming complex LEO or MEO satellite networks.
Image Compression With Learned Lifting-Based DWT and Learned Tree-Based Entropy Models
Authors: Ugur Berk Sahin, Fatih Kamisli
Date: 2022-12-07 00:00:00
Tasks: Image Compression

This paper explores learned image compression based on traditional and learned discrete wavelet transform (DWT) architectures and learned entropy models for coding DWT subband coefficients. A learned DWT is obtained through the lifting scheme with learned nonlinear predict and update filters. Several learned entropy models are proposed to exploit inter and intra-DWT subband coefficient dependencies, akin to traditional EZW, SPIHT, or EBCOT algorithms. Experimental results show that when the proposed learned entropy models are combined with traditional wavelet filters, such as the CDF 9/7 filters, compression performance that far exceeds that of JPEG2000 can be achieved. When the learned entropy models are combined with the learned DWT, compression performance increases further. The computations in the learned DWT and all entropy models, except one, can be simply parallelized, and the systems provide practical encoding and decoding times on GPUs.
Enabling Resilient and Real-Time Network Operations in Space: A Novel Multi-Layer Satellite Networking Scheme
Authors: Peng Hu
Date: 2022-12-07 00:00:00

Recently advanced low-Earth-orbit (LEO) satellite networks represented by large constellations and advanced payloads provide great promises for enabling high-quality Internet connectivity to any place on Earth. However, the traditional access-based approach to satellite operations cannot meet the pressing requirements of real-time, reliable, and resilient operations for LEO satellites. A new scheme is proposed based on multi-layer satellite networking considering the advanced Ka-band and optical communications payloads on a satellite platform. The proposed scheme can enable efficient and resilient message transmissions for critical telecommand and telemetry missions through different layers of satellite networks, which consist of LEO, medium-Earth-orbit (MEO), and geostationary (GEO) satellites. The proposed scheme is evaluated in a 24-hr satellite mission and shows superior performance improvements compared to the traditional operations approach.
EASpace: Enhanced Action Space for Policy Transfer
Authors: Zheng Zhang, Qingrui Zhang, Bo Zhu, Xiaohan Wang, Tianjiang Hu
Date: 2022-12-07 00:00:00
Tasks: Q-Learning, Transfer Learning

Formulating expert policies as macro actions promises to alleviate the long-horizon issue via structured exploration and efficient credit assignment. However, traditional option-based multi-policy transfer methods suffer from inefficient exploration of macro action's length and insufficient exploitation of useful long-duration macro actions. In this paper, a novel algorithm named EASpace (Enhanced Action Space) is proposed, which formulates macro actions in an alternative form to accelerate the learning process using multiple available sub-optimal expert policies. Specifically, EASpace formulates each expert policy into multiple macro actions with different execution {times}. All the macro actions are then integrated into the primitive action space directly. An intrinsic reward, which is proportional to the execution time of macro actions, is introduced to encourage the exploitation of useful macro actions. The corresponding learning rule that is similar to Intra-option Q-learning is employed to improve the data efficiency. Theoretical analysis is presented to show the convergence of the proposed learning rule. The efficiency of EASpace is illustrated by a grid-based game and a multi-agent pursuit problem. The proposed algorithm is also implemented in physical systems to validate its effectiveness.
Progressive Multi-Scale Self-Supervised Learning for Speech Recognition
Authors: Genshun Wan, Tan Liu, Hang Chen, Jia Pan, Cong Liu, Zhongfu Ye
Date: 2022-12-07 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Self-Supervised Learning, speech-recognition, Speech Recognition

Self-supervised learning (SSL) models have achieved considerable improvements in automatic speech recognition (ASR). In addition, ASR performance could be further improved if the model is dedicated to audio content information learning theoretically. To this end, we propose a progressive multi-scale self-supervised learning (PMS-SSL) method, which uses fine-grained target sets to compute SSL loss at top layer while uses coarse-grained target sets at intermediate layers. Furthermore, PMS-SSL introduces multi-scale structure into multi-head self-attention for better speech representation, which restricts the attention area into a large scope at higher layers while restricts the attention area into a small scope at lower layers. Experiments on Librispeech dataset indicate the effectiveness of our proposed method. Compared with HuBERT, PMS-SSL achieves 13.7% / 12.7% relative WER reduction on test other evaluation subsets respectively when fine-tuned on 10hours / 100hours subsets.
Improving trajectory localization accuracy via direction-of-arrival derivative estimation
Authors: Ruchi Pandey, Shreyas Jaiswal, Huy Phan, Santosh Nannuru
Date: 2022-12-07 00:00:00
Tasks: Direction of Arrival Estimation, Sound Source Localization

Sound source localization is crucial in acoustic sensing and monitoring-related applications. In this paper, we do a comprehensive analysis of improvement in sound source localization by combining the direction of arrivals (DOAs) with their derivatives which quantify the changes in the positions of sources over time. This study uses the SALSA-Lite feature with a convolutional recurrent neural network (CRNN) model for predicting DOAs and their first-order derivatives. An update rule is introduced to combine the predicted DOAs with the estimated derivatives to obtain the final DOAs. The experimental validation is done using TAU-NIGENS Spatial Sound Events (TNSSE) 2021 dataset. We compare the performance of the networks predicting DOAs with derivative vs. the one predicting only the DOAs at low SNR levels. The results show that combining the derivatives with the DOAs improves the localization accuracy of moving sources.
Selector-Enhancer: Learning Dynamic Selection of Local and Non-local Attention Operation for Speech Enhancement
Authors: Xinmeng Xu, Weiping tu, Yuhong Yang
Date: 2022-12-07 00:00:00
Tasks: Denoising, Reinforcement Learning (RL), Speech Enhancement

Attention mechanisms, such as local and non-local attention, play a fundamental role in recent deep learning based speech enhancement (SE) systems. However, natural speech contains many fast-changing and relatively brief acoustic events, therefore, capturing the most informative speech features by indiscriminately using local and non-local attention is challenged. We observe that the noise type and speech feature vary within a sequence of speech and the local and non-local operations can respectively extract different features from corrupted speech. To leverage this, we propose Selector-Enhancer, a dual-attention based convolution neural network (CNN) with a feature-filter that can dynamically select regions from low-resolution speech features and feed them to local or non-local attention operations. In particular, the proposed feature-filter is trained by using reinforcement learning (RL) with a developed difficulty-regulated reward that is related to network performance, model complexity, and "the difficulty of the SE task". The results show that our method achieves comparable or superior performance to existing approaches. In particular, Selector-Enhancer is potentially effective for real-world denoising, where the number and types of noise are varies on a single noisy mixture.
BC-VAD: A Robust Bone Conduction Voice Activity Detection
Authors: Niccolo' Polvani, Damien Ronssin, Milos Cernak
Date: 2022-12-06 00:00:00
Tasks: Action Detection, Activity Detection

Voice Activity Detection (VAD) is a fundamental module in many audio applications. Recent state-of-the-art VAD systems are often based on neural networks, but they require a computational budget that usually exceeds the capabilities of a small battery-operated device when preserving the performance of larger models. In this work, we rely on the input from a bone conduction microphone (BCM) to design an efficient VAD (BC-VAD) robust against residual non-stationary noises originating from the environment or speakers not wearing the BCM.We first show that a larger VAD system (58k parameters) achieves state-of-the-art results on a publicly available benchmark but fails when running on bone conduction signals. We then compare its variant BC-VAD (5k parameters and trained on BC data) with a baseline especially designed for a BCM and show that the proposed method achieves better performances under various metrics while keeping the realtime processing requirement for a microcontroller.
Self-Supervised Audio-Visual Speech Representations Learning By Multimodal Self-Distillation
Authors: Jing-Xuan Zhang, Genshun Wan, Zhen-Hua Ling, Jia Pan, Jianqing Gao, Cong Liu
Date: 2022-12-06 00:00:00
Tasks: Language Modeling, Language Modelling

In this work, we present a novel method, named AV2vec, for learning audio-visual speech representations by multimodal self-distillation. AV2vec has a student and a teacher module, in which the student performs a masked latent feature regression task using the multimodal target features generated online by the teacher. The parameters of the teacher model are a momentum update of the student. Since our target features are generated online, AV2vec needs no iteration step like AV-HuBERT and the total training time cost is reduced to less than one-fifth. We further propose AV2vec-MLM in this study, which augments AV2vec with a masked language model (MLM)-style loss using multitask learning. Our experimental results show that AV2vec achieved comparable performance to the AV-HuBERT baseline. When combined with an MLM-style loss, AV2vec-MLM outperformed baselines and achieved the best performance on the downstream tasks.
Sound emergence as a predictor of short-term annoyance from wind turbine noise
Authors: Elise Ruaud, Guillaume Dutilleux
Date: 2022-12-05 00:00:00

While sound emergence is used in several countries to regulate wind energy development, there is no published evidence that it is a relevant noise descriptor for this purpose. In the present work, we carried out two listening tests to evaluate the merits of sound emergence. Three definitions of sound emergence were considered: the one in ISO 1996-1, sound emergence under audibility condition $e_{UAC}$, and spectral emergence $e_{SP}$. We also considered the specific to residual ratio and loudness metrics. In each listening test, the sound stimuli consisted of 48 sound stimuli at 3 A-weighted sound pressure levels $\{30, 40, 50\}$~dB and 4 specific-to-residual ratios $\{-10, -5, 0, +5 \}$~dB. The results lead to the conclusion that short term annoyance is better predicted by the total sound pressure level than by sound emergence, whatever the definition considered for the latter, or than by the specific to residual ratio. Short-term annoyance is slightly better predicted by $e_{UAC}$ than by $e$, while $e$ is a better predictor than $e_{SP}$. The total sound pressure level and the loudness metrics performed similarly. Furthermore, the results provide evidence that sound emergence is a poor predictor of the audibility of wind turbine sounds.