pwc-archive/papers-with-abstracts huggingface dataset:
Rediscovering Ranganathan: A Prismatic View of His Life through the Knowledge Graph Spectrum
Authors: B. Dutta, S. Arzoo
Date: 2023-11-23 00:00:00
Tasks: Articles

The present study puts forward a novel biographical knowledge graph (KG) on Prof. S. R. Ranganathan, one of the pioneering figures in the Library and Information Science (LIS) domain. It has been found that most of the relevant facts about Ranganathan exist in a variety of resources (e.g., books, essays, journal articles, websites, blogs, etc.), offering information in a fragmented and piecemeal way. With this dedicated KG (henceforth known as RKG), we hope to furnish a 360-degree view of his life and achievements. To the best of our knowledge, such a dedicated representation is unparalleled in its scope and coverage: using state-of-the-art technology for anyone to openly access, use/re-use, and contribute. Inspired by Ranganathan's theories and ideas, the KG was developed using a "facet-based methodology" at two levels: in the identification of the vital biographical aspects and the development of the ontological model. Finally, with this study, we call for a community-driven effort to enhance the KG and pay homage to the Father of Library Science on the hundredth anniversary of his revitalizing the LIS domain through his enduring participation.
UAV-enabled Integrated Sensing and Communication: Tracking Design and Optimization
Authors: Yifan Jiang, Qingqing Wu, Wen Chen, Kaitao Meng
Date: 2024-01-08 00:00:00
Tasks: Integrated sensing and communication, ISAC, Object, Position

Integrated sensing and communications (ISAC) enabled by unmanned aerial vehicles (UAVs) is a promising technology to facilitate target tracking applications. In contrast to conventional UAV-based ISAC system designs that mainly focus on estimating the target position, the target velocity estimation also needs to be considered due to its crucial impacts on link maintenance and real-time response, which requires new designs on resource allocation and tracking scheme. In this paper, we propose an extended Kalman filtering-based tracking scheme for a UAV-enabled ISAC system where a UAV tracks a moving object and also communicates with a device attached to the object. Specifically, a weighted sum of predicted posterior Cram\'er-Rao bound (PCRB) for object relative position and velocity estimation is minimized by optimizing the UAV trajectory, where an efficient solution is obtained based on the successive convex approximation method. Furthermore, under a special case with the measurement mean square error (MSE), the optimal relative motion state is obtained and proved to keep a fixed elevation angle and zero relative velocity. Numerical results validate that the obtained solution to the predicted PCRB minimization can be approximated by the optimal relative motion state when predicted measurement MSE dominates the predicted PCRBs, as well as the effectiveness of the proposed tracking scheme. Moreover, three interesting trade-offs on system performance resulted from the fixed elevation angle are illustrated.
Assessing the Potential of Space-Time-Coding Metasurfaces for Sensing and Localization
Authors: Herman L. dos Santos, Martin Voigt Vejling, Taufik Abrão, Petar Popovski
Date: 2024-01-06 00:00:00
Tasks: ISAC

Intelligent metasurfaces are one of the favorite technologies for integrating sixth-generation (6G) networks, especially the reconfigurable intelligent surface (RIS) that has been extensively researched in various applications. In this context, a feature that deserves further exploration is the frequency scattering that occurs when the elements are periodically switched, referred to as Space-Time-Coding metasurface (STCM) topology. This type of topology causes impairments to the established communication methods by generating undesirable interference both in frequency and space, which is worsened when using wideband signals. Nevertheless, it has the potential to bring forward useful features for sensing and localization. This work exploits STCM sensing capabilities in target detection, localization, and classification using narrowband downlink pilot signals at the base station (BS). The results of this novel approach reveal the ability to retrieve a scattering point (SP) localization within the sub-centimeter and sub-decimeter accuracy depending on the SP position in space. We also analyze the associated detection and classification probabilities, which show reliable detection performance in the whole analyzed environment. In contrast, the classification is bounded by physical constraints, and we conclude that this method presents a promising approach for future integrated sensing and communications (ISAC) protocols by providing a tool to perform sensing and localization services using legacy communication signals.
Theoretical Framework for the Optimization of Microphone Array Configuration for Humanoid Robot Audition
Authors: Vladimir Tourbabin, Boaz Rafaely
Date: 2024-01-06 00:00:00

An important aspect of a humanoid robot is audition. Previous work has presented robot systems capable of sound localization and source segregation based on microphone arrays with various configurations. However, no theoretical framework for the design of these arrays has been presented. In the current paper, a design framework is proposed based on a novel array quality measure. The measure is based on the effective rank of a matrix composed of the generalized head related transfer functions (GHRTFs) that account for microphone positions other than the ears. The measure is shown to be theoretically related to standard array performance measures such as beamforming robustness and DOA estimation accuracy. Then, the measure is applied to produce sample designs of microphone arrays. Their performance is investigated numerically, verifying the advantages of array design based on the proposed theoretical framework.
Inverse Nonlinearity Compensation of Hyperelastic Deformation in Dielectric Elastomer for Acoustic Actuation
Authors: Jin Woo Lee, Gwang Seok An, Jeong-Yun Sun, Kyogu Lee
Date: 2024-01-08 00:00:00
Tasks: Numerical Integration

This paper presents an in-depth examination of the nonlinear deformation induced by dielectric actuation in pre-stressed ideal dielectric elastomers. A nonlinear ordinary differential equation that governs this deformation is formulated based on the hyperelastic model under dielectric stress. By means of numerical integration and neural network approximations, the relationship between voltage and stretch is established. Neural networks are utilized to approximate solutions for voltage-to-stretch and stretch-to-voltage transformations obtained via an explicit Runge-Kutta method. The efficacy of these approximations is illustrated by their use in compensating for nonlinearity through the waveshaping of the input signal. The comparative analysis demonstrates that the approximated solutions are more accurate than baseline methods, resulting in reduced harmonic distortions when dielectric elastomers are used as acoustic actuators. This study highlights the effectiveness of the proposed approach in mitigating nonlinearities and enhancing the performance of dielectric elastomers in acoustic actuation applications.
Creating Personalized Synthetic Voices from Articulation Impaired Speech Using Augmented Reconstruction Loss
Authors: Yusheng Tian, Jingyu Li, Tan Lee
Date: 2024-01-08 00:00:00

This research is about the creation of personalized synthetic voices for head and neck cancer survivors. It is focused particularly on tongue cancer patients whose speech might exhibit severe articulation impairment. Our goal is to restore normal articulation in the synthesized speech, while maximally preserving the target speaker's individuality in terms of both the voice timbre and speaking style. This is formulated as a task of learning from noisy labels. We propose to augment the commonly used speech reconstruction loss with two additional terms. The first term constitutes a regularization loss that mitigates the impact of distorted articulation in the training speech. The second term is a consistency loss that encourages correct articulation in the generated speech. These additional loss terms are obtained from frame-level articulation scores of original and generated speech, which are derived using a separately trained phone classifier. Experimental results on a real case of tongue cancer patient confirm that the synthetic voice achieves comparable articulation quality to unimpaired natural speech, while effectively maintaining the target speaker's individuality. Audio samples are available at https://myspeechproject.github.io/ArticulationRepair/.
LUPET: Incorporating Hierarchical Information Path into Multilingual ASR
Authors: Wei Liu, Jingyong Hou, Dong Yang, Muyong Cao, Tan Lee
Date: 2024-01-08 00:00:00
Tasks: Acoustic Unit Discovery, Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Toward high-performance multilingual automatic speech recognition (ASR), various types of linguistic information and model design have demonstrated their effectiveness independently. They include language identity (LID), phoneme information, language-specific processing modules, and cross-lingual self-supervised speech representation. It is expected that leveraging their benefits synergistically in a unified solution would further improve the overall system performance. This paper presents a novel design of a hierarchical information path, named LUPET, which sequentially encodes, from the shallow layers to deep layers, multiple aspects of linguistic and acoustic information at diverse granularity scales. The path starts from LID prediction, followed by acoustic unit discovery, phoneme sharing, and finally token recognition routed by a mixture-of-expert. ASR experiments are carried out on 10 languages in the Common Voice corpus. The results demonstrate the superior performance of LUPET as compared to the baseline systems. Most importantly, LUPET effectively mitigates the issue of performance compromise of high-resource languages with low-resource ones in the multilingual setting.
BS-PLCNet: Band-split Packet Loss Concealment Network with Multi-task Learning Framework and Multi-discriminators
Authors: Zihan Zhang, Jiayao Sun, Xianjun Xia, Chuanzeng Huang, Yijian Xiao, Lei Xie
Date: 2024-01-08 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Multi-Task Learning, Packet Loss Concealment, speech-recognition, Speech Recognition

Packet loss is a common and unavoidable problem in voice over internet phone (VoIP) systems. To deal with the problem, we propose a band-split packet loss concealment network (BS-PLCNet). Specifically, we split the full-band signal into wide-band (0-8kHz) and high-band (8-24kHz). The wide-band signals are processed by a gated convolutional recurrent network (GCRN), while the high-band counterpart is processed by a simple GRU network. To ensure high speech quality and automatic speech recognition (ASR) compatibility, multi-task learning (MTL) framework including fundamental frequency (f0) prediction, linguistic awareness, and multi-discriminators are used. The proposed approach tied for 1st place in the ICASSP 2024 PLC Challenge.
DDD: A Perceptually Superior Low-Response-Time DNN-based Declipper
Authors: Jayeon Yi, Junghyun Koo, Kyogu Lee
Date: 2024-01-08 00:00:00

Clipping is a common nonlinear distortion that occurs whenever the input or output of an audio system exceeds the supported range. This phenomenon undermines not only the perception of speech quality but also downstream processes utilizing the disrupted signal. Therefore, a real-time-capable, robust, and low-response-time method for speech declipping (SD) is desired. In this work, we introduce DDD (Demucs-Discriminator-Declipper), a real-time-capable speech-declipping deep neural network (DNN) that requires less response time by design. We first observe that a previously untested real-time-capable DNN model, Demucs, exhibits a reasonable declipping performance. Then we utilize adversarial learning objectives to increase the perceptual quality of output speech without additional inference overhead. Subjective evaluations on harshly clipped speech shows that DDD outperforms the baselines by a wide margin in terms of speech quality. We perform detailed waveform and spectral analyses to gain an insight into the output behavior of DDD in comparison to the baselines. Finally, our streaming simulations also show that DDD is capable of sub-decisecond mean response times, outperforming the state-of-the-art DNN approach by a factor of six.
Hyperbolic Distance-Based Speech Separation
Authors: Darius Petermann, Minje Kim
Date: 2024-01-07 00:00:00
Tasks: Speech Separation

In this work, we explore the task of hierarchical distance-based speech separation defined on a hyperbolic manifold. Based on the recent advent of audio-related tasks performed in non-Euclidean spaces, we propose to make use of the Poincar\'e ball to effectively unveil the inherent hierarchical structure found in complex speaker mixtures. We design two sets of experiments in which the distance-based parent sound classes, namely "near" and "far", can contain up to two or three speakers (i.e., children) each. We show that our hyperbolic approach is suitable for unveiling hierarchical structure from the problem definition, resulting in improved child-level separation. We further show that a clear correlation emerges between the notion of hyperbolic certainty (i.e., the distance to the ball's origin) and acoustic semantics such as speaker density, inter-source location, and microphone-to-speaker distance.
Theory and investigation of acoustic multiple-input multiple-output systems based on spherical arrays in a room
Authors: Hai Morgenstern, Boaz Rafaely, Franz Zotter
Date: 2024-01-07 00:00:00

Spatial attributes of room acoustics have been widely studied using microphone and loudspeaker arrays. However, systems that combine both arrays, referred to as multiple-input multiple-output (MIMO) systems, have only been studied to a limited degree in this context. These systems can potentially provide a powerful tool for room acoustics analysis due to the ability to simultaneously control both arrays. This paper offers a theoretical framework for the spatial analysis of enclosed sound fields using a MIMO system comprising spherical loudspeaker and microphone arrays. A system transfer function is formulated in matrix form for free-field conditions, and its properties are studied using tools from linear algebra. The system is shown to have unit-rank, regardless of the array types, and its singular vectors are related to the directions of arrival and radiation at the microphone and loudspeaker arrays, respectively. The formulation is then generalized to apply to rooms, using an image source method. In this case, the rank of the system is related to the number of significant reflections. The paper ends with simulation studies, which support the developed theory, and with an extensive reflection analysis of a room impulse response, using the platform of a MIMO system.
Multichannel AV-wav2vec2: A Framework for Learning Multichannel Multi-Modal Speech Representation
Authors: Qiushi Zhu, Jie Zhang, Yu Gu, Yuchen Hu, LiRong Dai
Date: 2024-01-07 00:00:00
Tasks: Audio-Visual Speech Recognition, Automatic Speech Recognition, Automatic Speech Recognition (ASR), Contrastive Learning, Self-Supervised Learning, speaker-diarization, Speaker Diarization, speech-recognition, Speech Recognition, Visual Speech Recognition

Self-supervised speech pre-training methods have developed rapidly in recent years, which show to be very effective for many near-field single-channel speech tasks. However, far-field multichannel speech processing is suffering from the scarcity of labeled multichannel data and complex ambient noises. The efficacy of self-supervised learning for far-field multichannel and multi-modal speech processing has not been well explored. Considering that visual information helps to improve speech recognition performance in noisy scenes, in this work we propose a multichannel multi-modal speech self-supervised learning framework AV-wav2vec2, which utilizes video and multichannel audio data as inputs. First, we propose a multi-path structure to process multichannel audio streams and a visual stream in parallel, with intra- and inter-channel contrastive losses as training targets to fully exploit the spatiotemporal information in multichannel speech data. Second, based on contrastive learning, we use additional single-channel audio data, which is trained jointly to improve the performance of speech representation. Finally, we use a Chinese multichannel multi-modal dataset in real scenarios to validate the effectiveness of the proposed method on audio-visual speech recognition (AVSR), automatic speech recognition (ASR), visual speech recognition (VSR) and audio-visual speaker diarization (AVSD) tasks.
Modal smoothing for analysis of room reflections measured with spherical microphone and loudspeaker arrays
Authors: Hai Morgenstern, Boaz Rafaely
Date: 2024-01-07 00:00:00

Spatial analysis of room acoustics is an ongoing research topic. Microphone arrays have been employed for spatial analyses with an important objective being the estimation of the direction-of-arrival (DOA) of direct sound and early room reflections using room impulse responses (RIRs). An optimal method for DOA estimation is the multiple signal classification algorithm. When RIRs are considered, this method typically fails due to the correlation of room reflections, which leads to rank deficiency of the cross-spectrum matrix. Preprocessing methods for rank restoration, which may involve averaging over frequency, for example, have been proposed exclusively for spherical arrays. However, these methods fail in the case of reflections with equal time delays, which may arise in practice and could be of interest. In this paper, a method is proposed for systems that combine a spherical microphone array and a spherical loudspeaker array, referred to as multiple-input multiple-output systems. This method, referred to as modal smoothing, exploits the additional spatial diversity for rank restoration and succeeds where previous methods fail, as demonstrated in a simulation study. Finally, combining modal smoothing with a preprocessing method is proposed in order to increase the number of DOAs that can be estimated using low-order spherical loudspeaker arrays.
Single-Microphone Speaker Separation and Voice Activity Detection in Noisy and Reverberant Environments
Authors: Renana Opochinsky, Mordehay Moradi, Sharon Gannot
Date: 2024-01-07 00:00:00
Tasks: Action Detection, Activity Detection, Decoder, Speaker Separation, Speech Separation

Speech separation involves extracting an individual speaker's voice from a multi-speaker audio signal. The increasing complexity of real-world environments, where multiple speakers might converse simultaneously, underscores the importance of effective speech separation techniques. This work presents a single-microphone speaker separation network with TF attention aiming at noisy and reverberant environments. We dub this new architecture as Separation TF Attention Network (Sep-TFAnet). In addition, we present a variant of the separation network, dubbed $ \text{Sep-TFAnet}^{\text{VAD}}$, which incorporates a voice activity detector (VAD) into the separation network. The separation module is based on a temporal convolutional network (TCN) backbone inspired by the Conv-Tasnet architecture with multiple modifications. Rather than a learned encoder and decoder, we use short-time Fourier transform (STFT) and inverse short-time Fourier transform (iSTFT) for the analysis and synthesis, respectively. Our system is specially developed for human-robotic interactions and should support online mode. The separation capabilities of $ \text{Sep-TFAnet}^{\text{VAD}}$ and Sep-TFAnet were evaluated and extensively analyzed under several acoustic conditions, demonstrating their advantages over competing methods. Since separation networks trained on simulated data tend to perform poorly on real recordings, we also demonstrate the ability of the proposed scheme to better generalize to realistic examples recorded in our acoustic lab by a humanoid robot. Project page: https://Sep-TFAnet.github.io
Spatial Reverberation and Dereverberation using an Acoustic Multiple-Input Multiple-Output System
Authors: Hai Morgenstern, Boaz Rafaely
Date: 2024-01-07 00:00:00

Methods are proposed for modifying the reverberation characteristics of sound fields in rooms by employing a loudspeaker with adjustable directivity, realized with a compact spherical loudspeaker array (SLA). These methods are based on minimization and maximization of clarity and direct-to-reverberant sound ratio. Significant modification of reverberation is achieved by these methods, as shown in simulation studies. The system under investigation includes a spherical microphone array and an SLA comprising a multiple-input multiple-output system. The robustness of these methods to system identification errors is also investigated. Finally, reverberation and dereverberation results are validated by a listening experiment.
Design framework for spherical microphone and loudspeaker arrays in a multiple-input multiple-output system
Authors: Hai Morgenstern, Boaz Rafaely, Markus Noisternig
Date: 2024-01-06 00:00:00

Spherical microphone arrays (SMAs) and spherical loudspeaker arrays (SLAs) facilitate the study of room acoustics due to the three-dimensional analysis they provide. More recently, systems that combine both arrays, referred to as multiple-input multiple-output (MIMO) systems, have been proposed due to the added spatial diversity they facilitate. The literature provides frameworks for designing SMAs and SLAs separately, including error analysis from which the operating frequency range (OFR) of an array is defined. However, such a framework does not exist for the joint design of a SMA and a SLA that comprise a MIMO system. This paper develops a design framework for MIMO systems based on a model that addresses errors and highlights the importance of a matched design. Expanding on a free-field assumption, errors are incorporated separately for each array and error bounds are defined, facilitating error analysis for the system. The dependency of the error bounds on the SLA and SMA parameters is studied and it is recommended that parameters should be chosen to assure matched OFRs of the arrays in MIMO system design. A design example is provided, demonstrating the superiority of a matched system over an unmatched system in the synthesis of directional room impulse responses.
IntervalMDP.jl: Accelerated Value Iteration for Interval Markov Decision Processes
Authors: Frederik Baymler Mathiesen, Morteza Lahijanian, Luca Laurenti
Date: 2024-01-08 00:00:00
Tasks: CPU, GPU

In this paper, we present IntervalMDP.jl, a Julia package for probabilistic analysis of interval Markov Decision Processes (IMDPs). IntervalMDP.jl facilitates the synthesis of optimal strategies and verification of IMDPs against reachability specifications and discounted reward properties. The library supports sparse matrices and is compatible with data formats from common tools for the analysis of probabilistic models, such as PRISM. A key feature of IntervalMDP.jl is that it presents both a multi-threaded CPU and a GPU-accelerated implementation of value iteration algorithms for IMDPs. In particular, IntervalMDP.jl takes advantage of the Julia type system and the inherently parallelizable nature of value iteration to improve the efficiency of performing analysis of IMDPs. On a set of examples, we show that IntervalMDP.jl substantially outperforms existing tools for verification and strategy synthesis for IMDPs in both computation time and memory consumption.