pwc-archive/papers-with-abstracts huggingface dataset:
A Hybrid CNN-BiLSTM Voice Activity Detector
Authors: Nicholas Wilkinson, Thomas Niesler
Date: 2021-03-05 00:00:00
Tasks: Action Detection, Activity Detection, Computational Efficiency

This paper presents a new hybrid architecture for voice activity detection (VAD) incorporating both convolutional neural network (CNN) and bidirectional long short-term memory (BiLSTM) layers trained in an end-to-end manner. In addition, we focus specifically on optimising the computational efficiency of our architecture in order to deliver robust performance in difficult in-the-wild noise conditions in a severely under-resourced setting. Nested k-fold cross-validation was used to explore the hyperparameter space, and the trade-off between optimal parameters and model size is discussed. The performance effect of a BiLSTM layer compared to a unidirectional LSTM layer was also considered. We compare our systems with three established baselines on the AVA-Speech dataset. We find that significantly smaller models with near optimal parameters perform on par with larger models trained with optimal parameters. BiLSTM layers were shown to improve accuracy over unidirectional layers by $\approx$2% absolute on average. With an area under the curve (AUC) of 0.951, our system outperforms all baselines, including a much larger ResNet system, particularly in difficult noise conditions.
Integration of deep learning with expectation maximization for spatial cue based speech separation in reverberant conditions
Authors: Sania Gul, Muhammad Salman Khan, Syed Waqar Shah
Date: 2021-02-26 00:00:00
Tasks: BIG-bench Machine Learning, blind source separation, Clustering, Deep Learning, Speech Separation

In this paper, we formulate a blind source separation (BSS) framework, which allows integrating U-Net based deep learning source separation network with probabilistic spatial machine learning expectation maximization (EM) algorithm for separating speech in reverberant conditions. Our proposed model uses a pre-trained deep learning convolutional neural network, U-Net, for clustering the interaural level difference (ILD) cues and machine learning expectation maximization (EM) algorithm for clustering the interaural phase difference (IPD) cues. The integrated model exploits the complementary strengths of the two approaches to BSS: the strong modeling power of supervised neural networks and the ease of unsupervised machine learning algorithms, whose few parameters can be estimated on as little as a single segment of an audio mixture. The results show an average improvement of 4.3 dB in signal to distortion ratio (SDR) and 4.3% in short time speech intelligibility (STOI) over the EM based source separation algorithm MESSL-GS (model-based expectation-maximization source separation and localization with garbage source) and 4.5 dB in SDR and 8% in STOI over deep learning convolutional neural network (U-Net) based speech separation algorithm SONET under the reverberant conditions ranging from anechoic to those mostly encountered in the real world.
Beam-Guided TasNet: An Iterative Speech Separation Framework with Multi-Channel Output
Authors: Hangting Chen, Yang Yi, Dang Feng, Pengyuan Zhang
Date: 2021-02-05 00:00:00
Tasks: blind source separation, Speech Separation

Time-domain audio separation network (TasNet) has achieved remarkable performance in blind source separation (BSS). Classic multi-channel speech processing framework employs signal estimation and beamforming. For example, Beam-TasNet links multi-channel convolutional TasNet (MC-Conv-TasNet) with minimum variance distortionless response (MVDR) beamforming, which leverages the strong modeling ability of data-driven network and boosts the performance of beamforming with an accurate estimation of speech statistics. Such integration can be viewed as a directed acyclic graph by accepting multi-channel input and generating multi-source output. In this paper, we design a "multi-channel input, multi-channel multi-source output" (MIMMO) speech separation system entitled "Beam-Guided TasNet", where MC-Conv-TasNet and MVDR can interact and promote each other more compactly under a directed cyclic flow. Specifically, the first stage uses Beam-TasNet to generate estimated single-speaker signals, which favors the separation in the second stage. The proposed framework facilitates iterative signal refinement with the guide of beamforming and seeks to reach the upper bound of the MVDR-based methods. Experimental results on the spatialized WSJ0-2MIX demonstrate that the Beam-Guided TasNet has achieved an SDR of 21.5 dB, exceeding the baseline Beam-TasNet by 4.1 dB under the same model size and narrowing the gap with the oracle signal-based MVDR to 2 dB.
WARP-Q: Quality Prediction For Generative Neural Speech Codecs
Authors: Wissam A. Jassim, Jan Skoglund, Michael Chinen, Andrew Hines
Date: 2021-02-20 00:00:00
Tasks: Dynamic Time Warping, Prediction

Good speech quality has been achieved using waveform matching and parametric reconstruction coders. Recently developed very low bit rate generative codecs can reconstruct high quality wideband speech with bit streams less than 3 kb/s. These codecs use a DNN with parametric input to synthesise high quality speech outputs. Existing objective speech quality models (e.g., POLQA, ViSQOL) do not accurately predict the quality of coded speech from these generative models underestimating quality due to signal differences not highlighted in subjective listening tests. We present WARP-Q, a full-reference objective speech quality metric that uses dynamic time warping cost for MFCC speech representations. It is robust to small perceptual signal changes. Evaluation using waveform matching, parametric and generative neural vocoder based codecs as well as channel and environmental noise shows that WARP-Q has better correlation and codec quality ranking for novel codecs compared to traditional metrics in addition to versatility for general quality assessment scenarios.
Model architectures to extrapolate emotional expressions in DNN-based text-to-speech
Authors: Katsuki Inoue, Sunao Hara, Masanobu Abe, Nobukatsu Hojo, Yusuke Ijima
Date: 2021-02-20 00:00:00
Tasks: text-to-speech, Text to Speech

This paper proposes architectures that facilitate the extrapolation of emotional expressions in deep neural network (DNN)-based text-to-speech (TTS). In this study, the meaning of "extrapolate emotional expressions" is to borrow emotional expressions from others, and the collection of emotional speech uttered by target speakers is unnecessary. Although a DNN has potential power to construct DNN-based TTS with emotional expressions and some DNN-based TTS systems have demonstrated satisfactory performances in the expression of the diversity of human speech, it is necessary and troublesome to collect emotional speech uttered by target speakers. To solve this issue, we propose architectures to separately train the speaker feature and the emotional feature and to synthesize speech with any combined quality of speakers and emotions. The architectures are parallel model (PM), serial model (SM), auxiliary input model (AIM), and hybrid models (PM&AIM and SM&AIM). These models are trained through emotional speech uttered by few speakers and neutral speech uttered by many speakers. Objective evaluations demonstrate that the performances in the open-emotion test provide insufficient information. They make a comparison with those in the closed-emotion test, but each speaker has their own manner of expressing emotion. However, subjective evaluation results indicate that the proposed models could convey emotional information to some extent. Notably, the PM can correctly convey sad and joyful emotions at a rate of >60%.
A Robust Maximum Likelihood Distortionless Response Beamformer based on a Complex Generalized Gaussian Distribution
Authors: Weixin Meng, Chengshi Zheng, XiaoDong Li
Date: 2021-02-19 00:00:00
Tasks: Speech Enhancement

For multichannel speech enhancement, this letter derives a robust maximum likelihood distortionless response beamformer by modeling speech sparse priors with a complex generalized Gaussian distribution, where we refer to as the CGGD-MLDR beamformer. The proposed beamformer can be regarded as a generalization of the minimum power distortionless response beamformer and its improved variations. For narrowband applications, we also reveal that the proposed beamformer reduces to the minimum dispersion distortionless response beamformer, which has been derived with the ${{\ell}_{p}}$-norm minimization. The mechanisms of the proposed beamformer in improving the robustness are clearly pointed out and experimental results show its better performance in PESQ improvement.
Polyp Segmentation in Colonoscopy Images using U-Net-MobileNetV2
Authors: Marcus V. L. Branch, Adriele S. Carvalho
Date: 2021-03-29 00:00:00

Colorectal cancer from the appearance of polyps that can be benign or malignant is one of the most fatal diseases in the world. To find these polyps in patients, colonoscopy is performed, which is a very efficient technique in this case. Clinically, detecting and segmenting these polyps in order to determine their presence or not is a very difficult process that requires a lot of time and experience from professionals, depending directly on these factors. Therefore, it becomes increasingly important to have an automatic, effective and reliable method of detecting and segmenting these polyps, making diagnoses faster and more accurate. In order to assist in the development of a method, we proposed the U-Net-MobileNetV2 model, which is the combination of two neural networks, where one acts as an encoder for the other and is responsible for learning image resources. Our experiments generated satisfactory results, demonstrating a good performance and good segmentation. U-Net-MobileNetV2 achieved a Dice Coefficient of 89.71% and an IoU of 81.64% for the Kvasir-SEG dataset, where both are higher than the results obtained by other state-of-the-art models.
A general framework for reversible data hiding in encrypted images by reserving room before encryption
Authors: Ammar Mohammadi
Date: 2021-03-28 00:00:00
Tasks: Blocking

In this paper a general framework to adopt different predictors for reversible data hiding in the encrypted image is presented. We propose innovative predictors that contribute more significantly than conventional ones results in accomplishing more payload. Reserving room before encryption (RRBE) is designated in the proposed scheme making possible to attain high embedding capacity. In RRBE procedure, pre-processing is allowed before image encryption. In our scheme, pre-processing comprises of three main steps: computing prediction-errors, blocking and labeling of the errors. By blocking, we obviate the need for lossless compression to when content-owner is not enthusiastic. Lossless compression is employed in recent state of the art schemes to improve payload. We surpass prior arts exploiting proper predictors, more efficient labeling procedure and blocking of the prediction-errors.
Selective Encryption of VVC Encoded Video Streams for the Internet of Video Things
Authors: Amir Fotovvat, Khan A. Wahid
Date: 2021-03-27 00:00:00

Visual sensors serve as a critical component of the Internet of Things (IoT). There is an ever-increasing demand for broad applications and higher resolutions of videos and cameras in smart homes and smart cities, such as in security cameras. To utilize this large volume of video data generated from networks of visual sensors for various machine vision applications, it needs to be compressed and securely transmitted over the Internet. H.266/VVC, as the new compression standard, brings the highest compression for visual data. To provide security along with high compression, a selective encryption method for hiding information of videos is presented for this new compression standard. Selective encryption methods can lower the computation overhead of the encryption while keeping the video bitstream format which is useful when the video goes into untrusted blocks such as transcoding or watermarking. Syntax elements that represent considerable information are selected for the encryption, i.e., luma Intra Prediction Modes (IPMs), Motion Vector Difference (MVD), and residual signs., then the results of the proposed method are investigated in terms of visual security and bit rate change. Our experiments show that the encrypted videos provide higher visual security compared to other similar works in previous standards, and integration of the presented encryption scheme into the VVC encoder has little impact on the bit rate efficiency (results in 2% to 3% bit rate increase).
A Comprehensive Review of Image Analysis Methods for Microorganism Counting: From Classical Image Processing to Deep Learning Approaches
Authors: Jiawei Zhang, Chen Li, Md Mamunur Rahaman, YuDong Yao, Pingli Ma, Jinghua Zhang, Xin Zhao, Tao Jiang, Marcin Grzegorzek
Date: 2021-03-25 00:00:00
Tasks: Articles, image-classification, Image Classification, Image Segmentation, Semantic Segmentation

Microorganisms such as bacteria and fungi play essential roles in many application fields, like biotechnique, medical technique and industrial domain. Microorganism counting techniques are crucial in microorganism analysis, helping biologists and related researchers quantitatively analyze the microorganisms and calculate their characteristics, such as biomass concentration and biological activity. However, traditional microorganism manual counting methods, such as plate counting method, hemocytometry and turbidimetry, are time-consuming, subjective and need complex operations, which are difficult to be applied in large-scale applications. In order to improve this situation, image analysis is applied for microorganism counting since the 1980s, which consists of digital image processing, image segmentation, image classification and suchlike. Image analysis-based microorganism counting methods are efficient comparing with traditional plate counting methods. In this article, we have studied the development of microorganism counting methods using digital image analysis. Firstly, the microorganisms are grouped as bacteria and other microorganisms. Then, the related articles are summarized based on image segmentation methods. Each part of the article is reviewed by methodologies. Moreover, commonly used image processing methods for microorganism counting are summarized and analyzed to find common technological points. More than 144 papers are outlined in this article. In conclusion, this paper provides new ideas for the future development trend of microorganism counting, and provides systematic suggestions for implementing integrated microorganism counting systems in the future. Researchers in other fields can refer to the techniques analyzed in this paper.
A Marker-free Head Tracker Using Vision-based Head Pose Estimation with Adaptive Kalman Filter
Authors: Zhongxu Hu, Chen Lv, Yanxin Zhou, Yiran Zhang, Wenhui Huang
Date: 2021-03-24 00:00:00
Tasks: Head Pose Estimation, Pose Estimation, Position

The immersion and the interaction are the important features of the driving simulator. To improve these characteristics, this paper proposes a low-cost and mark-less driver head tracking framework based on the head pose estimation model, which makes the view of the simulator can automatically align with the driver's head pose. The proposed method only uses the RGB camera without the other hardware or marker. To handle the error of the head pose estimation model, this paper proposes an adaptive Kalman Filter. By analyzing the error distribution of the estimation model and user experience, the proposed Kalman Filter includes the adaptive observation noise coefficient and loop closure module, which can adaptive moderate the smoothness of the curve and keep the curve stable near the initial position. The experiments show that the proposed method is feasible, and it can be used with different head pose estimation models.
Resonant Scanning Design and Control for Fast Spatial Sampling
Authors: Zhanghao Sun, Ronald Quan, Olav Solgaard
Date: 2021-03-24 00:00:00
Tasks: 3D Object Detection, object-detection, Object Detection

Two-dimensional, resonant scanners have been utilized in a large variety of imaging modules due to their compact form, low power consumption, large angular range, and high speed. However, resonant scanners have problems with non-optimal and inflexible scanning patterns and inherent phase uncertainty, which limit practical applications. Here we propose methods for optimized design and control of the scanning trajectory of two-dimensional resonant scanners under various physical constraints, including high frame-rate and limited actuation amplitude. First, we propose an analytical design rule for uniform spatial sampling. We demonstrate theoretically and experimentally that by including non-repeating scanning patterns, the proposed designs outperform previous designs in terms of scanning range and fill factor. Second, we show that we can create flexible scanning patterns that allow focusing on user-defined Regions-of-Interest (RoI) by modulation of the scanning parameters. The scanning parameters are found by an optimization algorithm. In simulations, we demonstrate the benefits of these designs with standard metrics and higher-level computer vision tasks (LiDAR odometry and 3D object detection). Finally, we experimentally implement and verify both unmodulated and modulated scanning modes using a two-dimensional, resonant MEMS scanner. Central to the implementations is high bandwidth monitoring of the phase of the angular scans in both dimensions. This task is carried out with a position-sensitive photodetector combined with high-bandwidth electronics, enabling fast spatial sampling at ~ 100Hz frame-rate.
Continuous monitoring of plant sub-cellular structural changes for plant and crop diseases detection by use of Intelligent Laser Speckle Classification (AI) technique
Authors: Ahmet Orun
Date: 2021-03-23 00:00:00
Tasks: Texture Classification

The continuous online monitoring of early signs of plant and crop diseases, at their early stages before a potential spread, is of high importance and necessitates multi-disciplinary techniques. Within this study a proposed technique achieves this goal by exploiting laser physics, textural image analysis, and AI for Shot hole disease. In this technique, specific laser light with a wavelength shorter than a sub-cellular component of an inspected plant, produces an interaction within the sub-cellular components and generates laser speckle patterns which can characterize those specific plant cells' features. The generated laser speckle image data then be quantized by texture analysis and classified by Bayesian networks. Such comparative methods manage to detect the differences at sub-cellular scales, such as nuclei modification, cellular shape, or size deformation, etc. for Shot hole disease with high classification accuracy between the healthy and diseased plants. The technique is capable of continuous online observation and monitoring of the plant or crop diseases via a wireless network at low instrumental cost and may replace the costly ground-truth field works
Microwave Breast Imaging via Neural Networks for Almost Real-time Applications
Authors: Michele Ambrosanio, Stefano Franceschini, Vito Pascazio, Fabio Baselice
Date: 2021-03-21 00:00:00
Tasks: valid

Conventional breast cancer imaging techniques are nowadays based on the use of ionising radiations or ultrasound waves for the inspection of breast areas. Nevertheless, these conventional techniques present some drawbacks related to patient safety, processing time and resolution issues. In this framework, microwave imaging can represent a valid alternative or a complementary technique compared to other conventional medical imaging modalities since it is safe (using non-ionising radiations), relatively cheap and more comfortable from patient point of view. Unfortunately, it is slow and computationally expensive, which strongly limit its use in clinical scenarios. In this paper, an artificial neural network for effective and almost real-time breast imaging is proposed. First, a realistic breast-like phantom generator was developed for training the network. Subsequently, numerical analyses have been conducted for the optimisation and the performance evaluation of the approach. The results seem very promising in terms of recovery performance as well as for the computation burden.
Probelab ReImager: An Open-Source Software for Streamlining Image Processing in an Electron Microscopy Laboratory
Authors: Noah Kraft, Anette von der Handt
Date: 2021-03-21 00:00:00

Providing high-quality electron images and hyperspectral X-ray maps is a focus of many modern electron microscopy laboratories. Nevertheless, further image processing and annotations are often needed to prepare them for publications and reports. For multi-user facilities, accessibility to processing software can be a limitation either through license costs or availability of processing stations. Open-source software running on multiple platforms allows for post-acquisition data processing in-lab or on user-owned devices. We developed Probelab ReImager to supersede our vendor-supplied acquisition software's exportation by being efficient and highly customizable. This article describes its main features and capabilities.
Motion Estimation for Optical Coherence Elastography Using Signal Phase and Intensity
Authors: Hossein Khodadadi, Orcun Goksel, Sabine Kling
Date: 2021-03-19 00:00:00
Tasks: Motion Estimation

Displacement estimation in optical coherence tomography (OCT) imaging is relevant for several potential applications, e.g. for optical coherence elastography (OCE) for corneal biomechanical characterization. Larger displacements may be resolved using correlation-based block matching techniques, which however are prone to signal de-correlation and imprecise at commonly desired sub-pixel resolutions. Phase-based tracking methods can estimate tiny sub-wavelength motion, but are not suitable for motion magnitudes larger than half the wavelength due to phase wrapping and the difficulty of any unwrapping due to noise. In this paper a robust OCT displacement estimation method is introduced by formulating tracking as an optimization problem that jointly penalizes intensity disparity, phase difference, and motion discontinuity. This is then solved using yynamic programming, utilizing both sub-wavelength-scale phase and pixel-scale intensity information from OCT imaging, while inherently seeking for the number of phase wraps. This allows for effectively tracking axial and lateral displacements, respectively, with sub-wavelength and pixel scale resolution. Results with tissue mimicking phantoms show that our proposed approach substantially outperforms conventional methods in terms of axial tracking precision, in particular for displacements exceeding half the imaging wavelength.