pwc-archive/papers-with-abstracts huggingface dataset:
Node-wise Localization of Graph Neural Networks
Authors: Zemin Liu, Yuan Fang, Chenghao Liu, Steven C. H. Hoi
Date: 2021-10-27 00:00:00
Tasks: Representation Learning

Graph neural networks (GNNs) emerge as a powerful family of representation learning models on graphs. To derive node representations, they utilize a global model that recursively aggregates information from the neighboring nodes. However, different nodes reside at different parts of the graph in different local contexts, making their distributions vary across the graph. Ideally, how a node receives its neighborhood information should be a function of its local context, to diverge from the global GNN model shared by all nodes. To utilize node locality without overfitting, we propose a node-wise localization of GNNs by accounting for both global and local aspects of the graph. Globally, all nodes on the graph depend on an underlying global GNN to encode the general patterns across the graph; locally, each node is localized into a unique model as a function of the global model and its local context. Finally, we conduct extensive experiments on four benchmark graphs, and consistently obtain promising performance surpassing the state-of-the-art GNNs.
IndoNLI: A Natural Language Inference Dataset for Indonesian
Authors: Rahmad Mahendra, Alham Fikri Aji, Samuel Louvan, Fahrurrozi Rahman, Clara Vania
Date: 2021-10-27 00:00:00
Proceeding: EMNLP 2021 11
Tasks: Natural Language Inference, Sentence, Spatial Reasoning, XLM-R

We present IndoNLI, the first human-elicited NLI dataset for Indonesian. We adapt the data collection protocol for MNLI and collect nearly 18K sentence pairs annotated by crowd workers and experts. The expert-annotated data is used exclusively as a test set. It is designed to provide a challenging test-bed for Indonesian NLI by explicitly incorporating various linguistic phenomena such as numerical reasoning, structural changes, idioms, or temporal and spatial reasoning. Experiment results show that XLM-R outperforms other pre-trained models in our data. The best performance on the expert-annotated data is still far below human performance (13.4% accuracy gap), suggesting that this test set is especially challenging. Furthermore, our analysis shows that our expert-annotated data is more diverse and contains fewer annotation artifacts than the crowd-annotated data. We hope this dataset can help accelerate progress in Indonesian NLP research.
Testing and Estimating Structural Breaks in Time Series and Panel Data in Stata
Authors: Jan Ditzen, Yiannis Karavias, Joakim Westerlund
Date: 2021-10-27 00:00:00
Tasks: Time Series, Time Series Analysis

Identifying structural change is a crucial step in analysis of time series and panel data. The longer the time span, the higher the likelihood that the model parameters have changed as a result of major disruptive events, such as the 2007--2008 financial crisis and the 2020 COVID--19 outbreak. Detecting the existence of breaks, and dating them is therefore necessary, not only for estimation purposes but also for understanding drivers of change and their effect on relationships. This article introduces a new community contributed command called xtbreak, which provides researchers with a complete toolbox for analysing multiple structural breaks in time series and panel data. xtbreak can detect the existence of breaks, determine their number and location, and provide break date confidence intervals. The new command is used to explore changes in the relationship between COVID--19 cases and deaths in the US, using both aggregate and state level data, and in the relationship between approval ratings and consumer confidence, using a panel of eight countries.
A Scalable Inference Method For Large Dynamic Economic Systems
Authors: Pratha Khandelwal, Philip Nadler, Rossella Arcucci, William Knottenbelt, Yi-Ke Guo
Date: 2021-10-27 00:00:00
Tasks: Bayesian Inference, BIG-bench Machine Learning, Interpretable Machine Learning

The nature of available economic data has changed fundamentally in the last decade due to the economy's digitisation. With the prevalence of often black box data-driven machine learning methods, there is a necessity to develop interpretable machine learning methods that can conduct econometric inference, helping policymakers leverage the new nature of economic data. We therefore present a novel Variational Bayesian Inference approach to incorporate a time-varying parameter auto-regressive model which is scalable for big data. Our model is applied to a large blockchain dataset containing prices, transactions of individual actors, analyzing transactional flows and price movements on a very granular level. The model is extendable to any dataset which can be modelled as a dynamical system. We further improve the simple state-space modelling by introducing non-linearities in the forward model with the help of machine learning architectures.
Forecasting with a Panel Tobit Model
Authors: Laura Liu, Hyungsik Roger Moon, Frank Schorfheide
Date: 2021-10-27 00:00:00
Tasks: model, Time Series, Time Series Analysis

We use a dynamic panel Tobit model with heteroskedasticity to generate forecasts for a large cross-section of short time series of censored observations. Our fully Bayesian approach allows us to flexibly estimate the cross-sectional distribution of heterogeneous coefficients and then implicitly use this distribution as prior to construct Bayes forecasts for the individual time series. In addition to density forecasts, we construct set forecasts that explicitly target the average coverage probability for the cross-section. We present a novel application in which we forecast bank-level loan charge-off rates for small banks.
The USS Trustee's risky strategy
Authors: Neil M Davies, Jackie Grant, Chin Yang Shapland
Date: 2021-10-27 00:00:00

How much risk, and what types of risk, is the Universities Superannuation Scheme (USS) taking? This is a critical question for universities across the UK and many of their employees. Will the fund have enough money to pay for all our pensions? Will it run out? Or is there a significant risk that we are collectively overpaying? In September 2021, David Miles and James Sefton, from Imperial College Business School, stepped into this vacuum, publishing 'How much risk is the USS taking?'. The paper presents important, accessible and highly readable analysis which estimates how likely the USS is to default over time. Their work is particularly relevant to the current UCU dispute with 69 employers over the benefit cuts that Universities UK (UUK) is planning to implement on the basis of the 2020 USS valuation. In this Brief, we assess the assumptions, replicate the results, explore further their model and consider potential extensions. We demonstrate that for a cautious model with reasonable assumptions for assets and asset growth, the fund has a less than 7% chance of defaulting for the duration that pensions promises are due, but a greater than 80% chance of being over funded by at least {\pounds}100bn, and nearly 50% chance of having over {\pounds}400bn. We offer warm thanks to David Miles and James Sefton for sharing their code and data, for their helpful conversations and clarification. Their analysis is infinitely clearer, better and more credible than anything the USS has produced. We hope this paper will be the beginning of more work in this area. All errors are our own.
Fisheries Management in Congested Waters: A Game-Theoretic Assessment of the East China Sea
Authors: Michael Macgregor Perry
Date: 2021-10-26 00:00:00
Tasks: Management

Fisheries in the East China Sea (ECS) face multiple concerning trends. Aside from depleted stocks caused by overfishing, illegal encroachments by fishermen from one nation into another's legal waters are a common occurrence. This behavior presumably could be stopped via strong monitoring, controls, and surveillance (MCS), but MCS is routinely rated below standards for nations bordering the ECS. This paper generalizes the ECS to a model of a congested maritime environment, defined as an environment where multiple nations can fish in the same waters with equivalent operating costs, and uses game-theoretic analysis to explain why the observed behavior persists in the ECS. The paper finds that nations in congested environments are incentivized to issue excessive quotas, which in turn tacitly encourages illegal fishing and extracts illegal rent from another's legal waters. This behavior couldn't persist in the face of strong MCS measures, and states are thus likewise incentivized to use poor MCS. A bargaining problem is analyzed to complement the noncooperative game, and a key finding is the nation with lower nonoperating costs has great leverage during the bargain.
Neural Analysis and Synthesis: Reconstructing Speech from Self-Supervised Representations
Authors: Hyeong-Seok Choi, Juheon Lee, Wansoo Kim, Jie Hwan Lee, Hoon Heo, Kyogu Lee
Date: 2021-10-27 00:00:00
Proceeding: NeurIPS 2021 12
Tasks: Voice Conversion

We present a neural analysis and synthesis (NANSY) framework that can manipulate voice, pitch, and speed of an arbitrary speech signal. Most of the previous works have focused on using information bottleneck to disentangle analysis features for controllable synthesis, which usually results in poor reconstruction quality. We address this issue by proposing a novel training strategy based on information perturbation. The idea is to perturb information in the original input signal (e.g., formant, pitch, and frequency response), thereby letting synthesis networks selectively take essential attributes to reconstruct the input signal. Because NANSY does not need any bottleneck structures, it enjoys both high reconstruction quality and controllability. Furthermore, NANSY does not require any labels associated with speech data such as text and speaker information, but rather uses a new set of analysis features, i.e., wav2vec feature and newly proposed pitch feature, Yingram, which allows for fully self-supervised training. Taking advantage of fully self-supervised training, NANSY can be easily extended to a multilingual setting by simply training it with a multilingual dataset. The experiments show that NANSY can achieve significant improvement in performance in several applications such as zero-shot voice conversion, pitch shift, and time-scale modification.
Exploring single-song autoencoding schemes for audio-based music structure analysis
Authors: Axel Marmoret, Jérémy E. Cohen, Frédéric Bimbot
Date: 2021-10-27 00:00:00

The ability of deep neural networks to learn complex data relations and representations is established nowadays, but it generally relies on large sets of training data. This work explores a "piece-specific" autoencoding scheme, in which a low-dimensional autoencoder is trained to learn a latent/compressed representation specific to a given song, which can then be used to infer the song structure. Such a model does not rely on supervision nor annotations, which are well-known to be tedious to collect and often ambiguous in Music Structure Analysis. We report that the proposed unsupervised auto-encoding scheme achieves the level of performance of supervised state-of-the-art methods with 3 seconds tolerance when using a Log Mel spectrogram representation on the RWC-Pop dataset.
Nonnegative Tucker Decomposition with Beta-divergence for Music Structure Analysis of Audio Signals
Authors: Axel Marmoret, Florian Voorwinden, Valentin Leplat, Jérémy E. Cohen, Frédéric Bimbot
Date: 2021-10-27 00:00:00
Tasks: Information Retrieval, Music Information Retrieval, Retrieval, tensor algebra, Tensor Decomposition

Nonnegative Tucker decomposition (NTD), a tensor decomposition model, has received increased interest in the recent years because of its ability to blindly extract meaningful patterns, in particular in Music Information Retrieval. Nevertheless, existing algorithms to compute NTD are mostly designed for the Euclidean loss. This work proposes a multiplicative updates algorithm to compute NTD with the beta-divergence loss, often considered a better loss for audio processing. We notably show how to implement efficiently the multiplicative rules using tensor algebra. Finally, we show on a music structure analysis task that unsupervised NTD fitted with beta-divergence loss outperforms earlier results obtained with the Euclidean loss.
Generalizing AUC Optimization to Multiclass Classification for Audio Segmentation With Limited Training Data
Authors: Pablo Gimeno, Victoria Mingote, Alfonso Ortega, Antonio Miguel, Eduardo Lleida
Date: 2021-10-27 00:00:00
Tasks: Segmentation

Area under the ROC curve (AUC) optimisation techniques developed for neural networks have recently demonstrated their capabilities in different audio and speech related tasks. However, due to its intrinsic nature, AUC optimisation has focused only on binary tasks so far. In this paper, we introduce an extension to the AUC optimisation framework so that it can be easily applied to an arbitrary number of classes, aiming to overcome the issues derived from training data limitations in deep learning solutions. Building upon the multiclass definitions of the AUC metric found in the literature, we define two new training objectives using a one-versus-one and a one-versus-rest approach. In order to demonstrate its potential, we apply them in an audio segmentation task with limited training data that aims to differentiate 3 classes: foreground music, background music and no music. Experimental results show that our proposal can improve the performance of audio segmentation systems significantly compared to traditional training criteria such as cross entropy.
Zero-shot Voice Conversion via Self-supervised Prosody Representation Learning
Authors: Shijun Wang, Dimche Kostadinov, Damian Borth
Date: 2021-10-27 00:00:00
Tasks: Disentanglement, Representation Learning, Voice Conversion

Voice Conversion (VC) for unseen speakers, also known as zero-shot VC, is an attractive research topic as it enables a range of applications like voice customizing, animation production, and others. Recent work in this area made progress with disentanglement methods that separate utterance content and speaker characteristics from speech audio recordings. However, many of these methods are subject to the leakage of prosody (e.g., pitch, volume), causing the speaker voice in the synthesized speech to be different from the desired target speakers. To prevent this issue, we propose a novel self-supervised approach that effectively learns disentangled pitch and volume representations that can represent the prosody styles of different speakers. We then use the learned prosodic representations as conditional information to train and enhance our VC model for zero-shot conversion. In our experiments, we show that our prosody representations are disentangled and rich in prosody information. Moreover, we demonstrate that the addition of our prosody representations improves our VC performance and surpasses state-of-the-art zero-shot VC performances.
Deep Learning For Prominence Detection In Children's Read Speech
Authors: Mithilesh Vaidya, Kamini Sabu, Preeti Rao
Date: 2021-10-27 00:00:00
Tasks: Deep Learning

The detection of perceived prominence in speech has attracted approaches ranging from the design of linguistic knowledge-based acoustic features to the automatic feature learning from suprasegmental attributes such as pitch and intensity contours. We present here, in contrast, a system that operates directly on segmented speech waveforms to learn features relevant to prominent word detection for children's oral fluency assessment. The chosen CRNN (convolutional recurrent neural network) framework, incorporating both word-level features and sequence information, is found to benefit from the perceptually motivated SincNet filters as the first convolutional layer. We further explore the benefits of the linguistic association between the prosodic events of phrase boundary and prominence with different multi-task architectures. Matching the previously reported performance on the same dataset of a random forest ensemble predictor trained on carefully chosen hand-crafted acoustic features, we evaluate further the possibly complementary information from hand-crafted acoustic and pre-trained lexical features.
Separating Long-Form Speech with Group-Wise Permutation Invariant Training
Authors: Wangyou Zhang, Zhuo Chen, Naoyuki Kanda, Shujie Liu, Jinyu Li, Sefik Emre Eskimez, Takuya Yoshioka, Xiong Xiao, Zhong Meng, Yanmin Qian, Furu Wei
Date: 2021-10-27 00:00:00
Tasks: Form, Speech Separation

Multi-talker conversational speech processing has drawn many interests for various applications such as meeting transcription. Speech separation is often required to handle overlapped speech that is commonly observed in conversation. Although the original utterancelevel permutation invariant training-based continuous speech separation approach has proven to be effective in various conditions, it lacks the ability to leverage the long-span relationship of utterances and is computationally inefficient due to the highly overlapped sliding windows. To overcome these drawbacks, we propose a novel training scheme named Group-PIT, which allows direct training of the speech separation models on the long-form speech with a low computational cost for label assignment. Two different speech separation approaches with Group-PIT are explored, including direct long-span speech separation and short-span speech separation with long-span tracking. The experiments on the simulated meeting-style data demonstrate the effectiveness of our proposed approaches, especially in dealing with a very long speech input.
Closing the Gap Between Time-Domain Multi-Channel Speech Enhancement on Real and Simulation Conditions
Authors: Wangyou Zhang, Jing Shi, Chenda Li, Shinji Watanabe, Yanmin Qian
Date: 2021-10-27 00:00:00
Tasks: Speech Enhancement, speech-recognition, Speech Recognition

The deep learning based time-domain models, e.g. Conv-TasNet, have shown great potential in both single-channel and multi-channel speech enhancement. However, many experiments on the time-domain speech enhancement model are done in simulated conditions, and it is not well studied whether the good performance can generalize to real-world scenarios. In this paper, we aim to provide an insightful investigation of applying multi-channel Conv-TasNet based speech enhancement to both simulation and real data. Our preliminary experiments show a large performance gap between the two conditions in terms of the ASR performance. Several approaches are applied to close this gap, including the integration of multi-channel Conv-TasNet into the beamforming model with various strategies, and the joint training of speech enhancement and speech recognition models. Our experiments on the CHiME-4 corpus show that our proposed approaches can greatly reduce the speech recognition performance discrepancy between simulation and real data, while preserving the strong speech enhancement capability in the frontend.
Temporal Knowledge Distillation for On-device Audio Classification
Authors: Kwanghee Choi, Martin Kersner, Jacob Morton, Buru Chang
Date: 2021-10-27 00:00:00
Tasks: Audio Classification, Classification, Event Detection, Keyword Spotting, Knowledge Distillation

Improving the performance of on-device audio classification models remains a challenge given the computational limits of the mobile environment. Many studies leverage knowledge distillation to boost predictive performance by transferring the knowledge from large models to on-device models. However, most lack a mechanism to distill the essence of the temporal information, which is crucial to audio classification tasks, or similar architecture is often required. In this paper, we propose a new knowledge distillation method designed to incorporate the temporal knowledge embedded in attention weights of large transformer-based models into on-device models. Our distillation method is applicable to various types of architectures, including the non-attention-based architectures such as CNNs or RNNs, while retaining the original network architecture during inference. Through extensive experiments on both an audio event detection dataset and a noisy keyword spotting dataset, we show that our proposed method improves the predictive performance across diverse on-device architectures.
TA-Net: Topology-Aware Network for Gland Segmentation
Authors: Haotian Wang, Min Xian, Aleksandar Vakanski
Date: 2021-10-27 00:00:00
Tasks: Instance Segmentation, Segmentation, Semantic Segmentation

Gland segmentation is a critical step to quantitatively assess the morphology of glands in histopathology image analysis. However, it is challenging to separate densely clustered glands accurately. Existing deep learning-based approaches attempted to use contour-based techniques to alleviate this issue but only achieved limited success. To address this challenge, we propose a novel topology-aware network (TA-Net) to accurately separate densely clustered and severely deformed glands. The proposed TA-Net has a multitask learning architecture and enhances the generalization of gland segmentation by learning shared representation from two tasks: instance segmentation and gland topology estimation. The proposed topology loss computes gland topology using gland skeletons and markers. It drives the network to generate segmentation results that comply with the true gland topology. We validate the proposed approach on the GlaS and CRAG datasets using three quantitative metrics, F1-score, object-level Dice coefficient, and object-level Hausdorff distance. Extensive experiments demonstrate that TA-Net achieves state-of-the-art performance on the two datasets. TA-Net outperforms other approaches in the presence of densely clustered glands.
PL-Net: Progressive Learning Network for Medical Image Segmentation
Authors: Junlong Cheng, Chengrui Gao, Hongchun Lu, Zhangqiang Ming, Yong Yang, Min Zhu
Date: 2021-10-27 00:00:00
Tasks: Image Segmentation, Medical Image Analysis, Medical Image Segmentation, Segmentation, Semantic Segmentation

In recent years, segmentation methods based on deep convolutional neural networks (CNNs) have made state-of-the-art achievements for many medical analysis tasks. However, most of these approaches improve performance by optimizing the structure or adding new functional modules of the U-Net, which ignoring the complementation and fusion of the coarse-grained and fine-grained semantic information. To solve the above problems, we propose a medical image segmentation framework called progressive learning network (PL-Net), which includes internal progressive learning (IPL) and external progressive learning (EPL). PL-Net has the following advantages: (1) IPL divides feature extraction into two "steps", which can mix different size receptive fields and capture semantic information from coarse to fine granularity without introducing additional parameters; (2) EPL divides the training process into two "stages" to optimize parameters, and realizes the fusion of coarse-grained information in the previous stage and fine-grained information in the latter stage. We evaluate our method in different medical image analysis tasks, and the results show that the segmentation performance of PL-Net is better than the state-of-the-art methods of U-Net and its variants.