pwc-archive/papers-with-abstracts huggingface dataset:
Audio-Visual Speech and Gesture Recognition by Sensors of Mobile Devices
Authors: Dmitry Ryumin, Denis Ivanko, Elena Ryumina
Date: 2023-02-17 00:00:00
Proceeding: Sensors 2023 2
Tasks: Audio-Visual Speech Recognition, Gesture Recognition, Lip Reading, Sign Language Recognition, speech-recognition, Speech Recognition, Visual Speech Recognition

Audio-visual speech recognition (AVSR) is one of the most promising solutions for reliable speech recognition, particularly when audio is corrupted by noise. Additional visual information can be used for both automatic lip-reading and gesture recognition. Hand gestures are a form of non-verbal communication and can be used as a very important part of modern human–computer interaction systems. Currently, audio and video modalities are easily accessible by sensors of mobile devices. However, there is no out-of-the-box solution for automatic audio-visual speech and gesture recognition. This study introduces two deep neural network-based model architectures: one for AVSR and one for gesture recognition. The main novelty regarding audio-visual speech recognition lies in fine-tuning strategies for both visual and acoustic features and in the proposed end-to-end model, which considers three modality fusion approaches: prediction-level, feature-level, and model-level. The main novelty in gesture recognition lies in a unique set of spatio-temporal features, including those that consider lip articulation information. As there are no available datasets for the combined task, we evaluated our methods on two different large-scale corpora—LRW and AUTSL—and outperformed existing methods on both audio-visual speech recognition and gesture recognition tasks. We achieved AVSR accuracy for the LRW dataset equal to 98.76% and gesture recognition rate for the AUTSL dataset equal to 98.56%. The results obtained demonstrate not only the high performance of the proposed methodology, but also the fundamental possibility of recognizing audio-visual speech and gestures by sensors of mobile devices.
Adverse weather amplifies social media activity
Authors: Kelton Minor, Esteban Moro, Nick Obradovich
Date: 2023-02-16 00:00:00

Humanity spends an increasing proportion of its time interacting online. Scholars are intensively investigating the societal drivers and resultant impacts of this collective shift in our allocation of time and attention. Yet, the external factors that regularly shape online behavior remain markedly understudied. Do environmental factors alter rates of online activity? Here we show that adverse meteorological conditions markedly increase social media use in the United States. To do so, we employ climate econometric methods alongside over three and a half billion social media posts from tens of millions of individuals from both Facebook and Twitter between 2009 and 2016. We find that more extreme temperatures and added precipitation each independently amplify social media activity. Weather that is adverse on both the temperature and precipitation dimensions produces markedly larger increases in social media activity. On average across both platforms, compared to the temperate weather baseline, days colder than -5{\deg}C with 1.5-2cm of precipitation elevate social media activity by 35%. This effect is nearly three times the typical increase in social media activity observed on New Year's Eve in New York City. We observe meteorological effects on social media participation at both the aggregate and individual level, even accounting for individual-specific, temporal, and location-specific potential confounds.
Joint Data Association, NLOS Mitigation, and Clutter Suppression for Networked Device-Free Sensing in 6G Cellular Network
Authors: Qin Shi, Liang Liu, Shuowen Zhang
Date: 2023-02-16 00:00:00
Tasks: Integrated sensing and communication, ISAC

Recently, there is a growing interest in achieving integrated sensing and communication (ISAC) in the sixth-generation (6G) cellular network. Inspired by this trend and the success of cooperative communication in cloud radio access network, this paper considers a networked device-free sensing architecture based on base station (BS) cooperation to transform the cellular network into a huge sensor that can provide ubiquitous and high-performance sensing services. Under this framework, the BSs first transmit the downlink communication signals to the mobile users and then estimate the range information of the targets based on their echoes. Next, a central processor collects the range information from all the BSs via the fronthaul links and localizes each target based on its distances to various BSs. To enable the above strategy in the 6G network, we will perform joint data association, non-line-of-sight (NLOS) mitigation, and clutter suppression, such that the central processor is able to find out the useful range estimations extracted from the line-of-sight (LOS) paths and match them to the right targets for localization. Numerical results show that our interested networked device-free sensing scheme for the 6G network can localize the targets with high accuracy in the challenging multi-path propagation environment.
Bidirectional UWB Localization: A Review on an Elastic Positioning Scheme for GNSS-deprived Zones
Authors: Cung Lian Sang, Michael Adams, Marc Hesse, Ulrich Rückert
Date: 2023-02-15 00:00:00

A bidirectional Ultra-Wideband (UWB) localization scheme is one of the three widely adopted design integration processes commonly used in time-based UWB positioning systems. The key property of bidirectional UWB localization is its ability to serve both navigation and tracking tasks within a single localization scheme on demand. Traditionally, navigation and tracking in wireless localization systems were treated as separate entities due to distinct applicable use-cases and methodological needs in each implementation process. Therefore, the ability to flexibly or elastically combine two unique positioning perspectives (navigation and tracking) within a single scheme can be regarded as a paradigm shift in the way location-based services are conventionally observed. This article reviews the mentioned bidirectional UWB localization from the perspective of a flexible and versatile positioning topology and highlights its potential in the field. In this regard, the article comprehensively describes the complete system model of the bidirectional UWB localization scheme using modular processes. It also discusses the demonstrative evaluation of two system integration processes and conducts a SWOT (Strengths, Weaknesses, Opportunities, and Threats) analysis of the scheme. Furthermore, the prospect of the presented bidirectional localization scheme for achieving precise location estimation in 5G/6G wireless mobile networks, as well as in Wi-Fi fine-time measurement-based positioning systems was briefly discussed.
Fast and Blind Speech Copy-Move Detection and Localization in Noise
Authors: Dong Yang, Mingle Liu, Muyong Cao
Date: 2023-02-15 00:00:00
Tasks: Computational Efficiency, Dynamic Time Warping

Copy-move forgery on speech (CMF), coupled with post-processing techniques, presents a great challenge to the forensic detection and localization of tampered areas. Most of the existing CMF detection approaches necessitate pre-segmentation of speech to facilitate similarity calculations among these segments. However, these approaches usually suffer from the problems of uncontrollable computational complexity and sensitivity to the presence of a word that is read multiple times within a speech recording. To address these issues, we propose a local feature tensors-based CMF detection algorithm that can transform duplicate detection and localization problems into a special tensor-matching procedure, accompanied by complete theoretical analysis as support. Through extensive experimentation, we have demonstrated that our method exhibits computational efficiency and robustness against post-processing techniques. Notably, it can effectively and blindly detect tampered segments, even those as short as a fractional second. These advantages highlight the promising potential of our approach for practical applications.
A Comparative Predicting Stock Prices using Heston and Geometric Brownian Motion Models
Authors: H. T. Shehzad, M. A. Anwar, M. Razzaq
Date: 2023-02-15 00:00:00
Tasks: LEMMA, Stock Market Prediction

This paper presents a novel approach to predicting stock prices using technical analysis. By utilizing Ito's lemma and Euler-Maruyama methods, the researchers develop Heston and Geometric Brownian Motion models that take into account volatility, interest rate, and historical stock prices to generate predictions. The results of the study demonstrate that these models are effective in accurately predicting stock prices and outperform commonly used statistical indicators. The authors conclude that this technical analysis-based method offers a promising solution for stock market prediction.
Lattice piecewise affine approximation of explicit nonlinear model predictive control with application to trajectory tracking of mobile robot
Authors: Kangbo Wang, Kaijie Zhang, Yating Huang, Jun Xu
Date: 2023-02-16 00:00:00
Tasks: Model Predictive Control

To promote the widespread use of mobile robots in diverse fields, the performance of trajectory tracking must be ensured. To address the constraints and nonlinear features associated with mobile robot systems, we apply nonlinear model predictive control (MPC) to realize the trajectory tracking of mobile robots. Specifically, to alleviate the online computational complexity of nonlinear MPC, this paper devises a lattice piecewise affine (PWA) approximation method that can approximate both the nonlinear system and control law of explicit nonlinear MPC. The kinematic model of the mobile robot is successively linearized along the trajectory to obtain a linear time-varying description of the system, which is then expressed using a lattice PWA model. Subsequently, the nonlinear MPC problem can be transformed into a series of linear MPC problems. Furthermore, to reduce the complexity of online calculation of multiple linear MPC problems, we approximate the optimal solution of the linear MPC by using the lattice PWA model. That is, for different sampling states, the optimal control inputs are obtained, and lattice PWA approximations are constructed for the state control pairs. Simulations are performed to evaluate the performance of our method in comparison with the linear MPC and explicit linear MPC frameworks. The results show that compared with the explicit linear MPC, our method has a higher online computing speed and can decrease the offline computing time without significantly increasing the tracking error.
Speech Enhancement with Multi-granularity Vector Quantization
Authors: Xiao-Ying Zhao, Qiu-Shi Zhu, Jie Zhang
Date: 2023-02-16 00:00:00
Tasks: Denoising, Quantization, Speech Denoising, Speech Enhancement

With advances in deep learning, neural network based speech enhancement (SE) has developed rapidly in the last decade. Meanwhile, the self-supervised pre-trained model and vector quantization (VQ) have achieved excellent performance on many speech-related tasks, while they are less explored on SE. As it was shown in our previous work that utilizing a VQ module to discretize noisy speech representations is beneficial for speech denoising, in this work we therefore study the impact of using VQ at different layers with different number of codebooks. Different VQ modules indeed enable to extract multiple-granularity speech features. Following an attention mechanism, the contextual features extracted by a pre-trained model are fused with the local features extracted by the encoder, such that both global and local information are preserved to reconstruct the enhanced speech. Experimental results on the Valentini dataset show that the proposed model can improve the SE performance, where the impact of choosing pre-trained models is also revealed.
DeepSpace: Dynamic Spatial and Source Cue Based Source Separation for Dialog Enhancement
Authors: Aaron Master, Lie Lu, Jonas Samuelsson, Heidi-Maria Lehtonen, Scott Norcross, Nathan Swedlow, Audrey Howard
Date: 2023-02-16 00:00:00
Tasks: Denoising

Dialog Enhancement (DE) is a feature which allows a user to increase the level of dialog in TV or movie content relative to non-dialog sounds. When only the original mix is available, DE is "unguided," and requires source separation. In this paper, we describe the DeepSpace system, which performs source separation using both dynamic spatial cues and source cues to support unguided DE. Its technologies include spatio-level filtering (SLF) and deep-learning based dialog classification and denoising. Using subjective listening tests, we show that DeepSpace demonstrates significantly improved overall performance relative to state-of-the-art systems available for testing. We explore the feasibility of using existing automated metrics to evaluate unguided DE systems.
Multi-Channel Target Speaker Extraction with Refinement: The WavLab Submission to the Second Clarity Enhancement Challenge
Authors: Samuele Cornell, Zhong-Qiu Wang, Yoshiki Masuyama, Shinji Watanabe, Manuel Pariente, Nobutaka Ono
Date: 2023-02-15 00:00:00
Tasks: Speaker Separation, Speech Enhancement, Target Speaker Extraction

This paper describes our submission to the Second Clarity Enhancement Challenge (CEC2), which consists of target speech enhancement for hearing-aid (HA) devices in noisy-reverberant environments with multiple interferers such as music and competing speakers. Our approach builds upon the powerful iterative neural/beamforming enhancement (iNeuBe) framework introduced in our recent work, and this paper extends it for target speaker extraction. We therefore name the proposed approach as iNeuBe-X, where the X stands for extraction. To address the challenges encountered in the CEC2 setting, we introduce four major novelties: (1) we extend the state-of-the-art TF-GridNet model, originally designed for monaural speaker separation, for multi-channel, causal speech enhancement, and large improvements are observed by replacing the TCNDenseNet used in iNeuBe with this new architecture; (2) we leverage a recent dual window size approach with future-frame prediction to ensure that iNueBe-X satisfies the 5 ms constraint on algorithmic latency required by CEC2; (3) we introduce a novel speaker-conditioning branch for TF-GridNet to achieve target speaker extraction; (4) we propose a fine-tuning step, where we compute an additional loss with respect to the target speaker signal compensated with the listener audiogram. Without using external data, on the official development set our best model reaches a hearing-aid speech perception index (HASPI) score of 0.942 and a scale-invariant signal-to-distortion ratio improvement (SI-SDRi) of 18.8 dB. These results are promising given the fact that the CEC2 data is extremely challenging (e.g., on the development set the mixture SI-SDR is -12.3 dB). A demo of our submitted system is available at WAVLab CEC2 demo.
Confidence Score Based Speaker Adaptation of Conformer Speech Recognition Systems
Authors: Jiajun Deng, Xurong Xie, Tianzi Wang, Mingyu Cui, Boyang Xue, Zengrui Jin, Guinan Li, Shujie Hu, Xunying Liu
Date: 2023-02-15 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Language Modelling, Sensitivity, speech-recognition, Speech Recognition

Speaker adaptation techniques provide a powerful solution to customise automatic speech recognition (ASR) systems for individual users. Practical application of unsupervised model-based speaker adaptation techniques to data intensive end-to-end ASR systems is hindered by the scarcity of speaker-level data and performance sensitivity to transcription errors. To address these issues, a set of compact and data efficient speaker-dependent (SD) parameter representations are used to facilitate both speaker adaptive training and test-time unsupervised speaker adaptation of state-of-the-art Conformer ASR systems. The sensitivity to supervision quality is reduced using a confidence score-based selection of the less erroneous subset of speaker-level adaptation data. Two lightweight confidence score estimation modules are proposed to produce more reliable confidence scores. The data sparsity issue, which is exacerbated by data selection, is addressed by modelling the SD parameter uncertainty using Bayesian learning. Experiments on the benchmark 300-hour Switchboard and the 233-hour AMI datasets suggest that the proposed confidence score-based adaptation schemes consistently outperformed the baseline speaker-independent (SI) Conformer model and conventional non-Bayesian, point estimate-based adaptation using no speaker data selection. Similar consistent performance improvements were retained after external Transformer and LSTM language model rescoring. In particular, on the 300-hour Switchboard corpus, statistically significant WER reductions of 1.0%, 1.3%, and 1.4% absolute (9.5%, 10.9%, and 11.3% relative) were obtained over the baseline SI Conformer on the NIST Hub5'00, RT02, and RT03 evaluation sets respectively. Similar WER reductions of 2.7% and 3.3% absolute (8.9% and 10.2% relative) were also obtained on the AMI development and evaluation sets.
A Frequency Domain Analysis of Slow Coherency in Networked Systems
Authors: Hancheng Min, Richard Pates, Enrique Mallada
Date: 2023-02-16 00:00:00

Network coherence generally refers to the emergence of simple aggregated dynamical behaviours, despite heterogeneity in the dynamics of the subsystems that constitute the network. In this paper, we develop a general frequency domain framework to analyze and quantify the level of network coherence that a system exhibits by relating coherence with a low-rank property of the system's input-output response. More precisely, for a networked system with linear dynamics and coupling, we show that, as the network's \emph{effective algebraic connectivity} grows, the system transfer matrix converges to a rank-one transfer matrix representing the coherent behavior. Interestingly, the non-zero eigenvalue of such a rank-one matrix is given by the harmonic mean of individual nodal dynamics, and we refer to it as the coherent dynamics. Our analysis unveils the frequency-dependent nature of coherence and a non-trivial interplay between dynamics and network topology. We further show that many networked systems can exhibit similar coherent behavior by establishing a concentration result in a setting with randomly chosen individual nodal dynamics.
Learning a Multi-Agent Controller for Shared Energy Storage System
Authors: Ruohong Liu, Yize Chen
Date: 2023-02-16 00:00:00
Tasks: Multi-agent Reinforcement Learning, Scheduling

Deployment of shared energy storage systems (SESS) allows users to use the stored energy to meet their own energy demands while saving energy costs without installing private energy storage equipment. In this paper, we consider a group of building users in the community with SESS, and each user can schedule power injection from the grid as well as SESS according to their demand and real-time electricity price to minimize energy cost and meet energy demand simultaneously. SESS is encouraged to charge when the price is low, thus providing as much energy as possible for users while achieving energy savings. However, due to the complex dynamics of buildings and real-time external signals, it is a challenging task to find high-performance power dispatch decisions in real-time. By designing a multi-agent reinforcement learning framework with state-aware reward functions, SESS and users can realize power scheduling to meet the users' energy demand and SESS's charging/discharging balance without additional communication, so as to achieve energy optimization. Compared with the baseline approach without the participation of the SESS, the energy cost is saved by around 2.37% to 21.58%.
On-chip fully reconfigurable Artificial Neural Network in 16 nm FinFET for Positron Emission Tomography
Authors: Andrada Muntean, Yonatan Shoshan, Slava Yuzhaninov, Emanuele Ripiccini, Claudio Bruschini, Alexander Fish, Edoardo Charbon
Date: 2023-02-16 00:00:00

Smarty is a fully-reconfigurable on-chip feed-forward artificial neural network (ANN) with ten integrated time-to-digital converters (TDCs) designed in a 16 nm FinFET CMOS technology node. The integration of TDCs together with an ANN aims to reduce system complexity and minimize data throughput requirements in positron emission tomography (PET) applications. The TDCs have an average LSB of 53.5 ps. The ANN is fully reconfigurable, the user being able to change its topology as desired within a set of constraints. The chip can execute 363 MOPS with a maximum power consumption of 1.9 mW, for an efficiency of 190 GOPS/W. The system performance was tested in a coincidence measurement setup interfacing Smarty with two groups of five 4 mm x 4 mm analog silicon photomultipliers (A-SiPMs) used as inputs for the TDCs. The ANN successfully distinguished between six different positions of a radioactive source placed between the two photodetector arrays by solely using the TDC timestamps.
A Model Predictive Control Framework for Improving Risk-Tolerance of Manufacturing Systems
Authors: Mostafa Tavakkoli Anbarani, Efe C. Balta, Rômulo Meira-Góes, Ilya Kovalenko
Date: 2023-02-16 00:00:00
Tasks: Model Predictive Control

The need for control strategies that can address dynamic system uncertainty is becoming increasingly important. In this work, we propose a Model Predictive Control by quantifying the risk of failure in our system model. The proposed control scheme uses a Priced Timed Automata representation of the manufacturing system to promote the fail-safe operation of systems under uncertainties. The proposed method ensures that in case of unforeseen failure(s), the optimization-based control strategy can still achieve the manufacturing system objective. In addition, the proposed strategy establishes a trade-off between minimizing the cost and reducing failure risk to allow the manufacturing system to function effectively in the presence of uncertainties. An example from manufacturing systems is presented to show the application of the proposed control strategy.
DC Microgrid based on Battery, Photovoltaic, and fuel Cells; Design and Control
Authors: Akram Muntaser, Abdurazag Saide, Hussin Ragb, Ibrahim Elwarfalli
Date: 2022-12-15 00:00:00

Microgrids offer flexibility in power generation in a way of using multiple renewable energy sources. In the past few years, microgrids become a very active research area in terms of design and control strategies. Most of the microgrids use DC/DC converters to connect renewable energy sources to the load. In this paper, the simulation model of a DC microgrid with three different energy sources (Lithium-ion battery (LIB), photovoltaic (PV) array, and fuel cell) and external variant power load is built with MATLAB/Simulink and the simulative results show that the stability of DC microgrid can be guaranteed by the proposed maximum power point controller MPPT. The three energy sources are connected to the load through DC/DC converters, one for each. This type of topology ensures protection for each energy source as well as optimum stability at the load.