pwc-archive/papers-with-abstracts huggingface dataset:
Leadership and Institutional Reforms
Authors: Matata Ponyo Mapon, Jean-Paul K. Tsasa
Date: 2021-01-21 00:00:00

Large-scale institutional changes require strong commitment and involvement of all stakeholders. We use the standard framework of cooperative game theory developed by Ichiishi (1983, pp. 78-149) to: (i) establish analytically the difference between policy maker and political leader; (ii) formally study interactions between a policy maker and his followers; (iii) examine the role of leadership in the implementation of structural reforms. We show that a policy maker can be both partisan and non-partisan, while a political leader can only be non-partisan. Following this distinction, we derive the probability of success of an institutional change, as well as the nature of the gain that such a change would generate on the beneficiary population. Based on the restrictions of this simple mathematical model and using some evidence from the Congolese experience between 2012 and 2016, we show that institutional changes can indeed benefit the majority of the population, when policy makers are truly partisan.
Multi-sense embeddings through a word sense disambiguation process
Authors: Terry Ruas, William Grosky, Akiko Aizawa
Date: 2021-01-21 00:00:00
Tasks: Natural Language Understanding, Word Embeddings, Word Sense Disambiguation, Word Similarity

Natural Language Understanding has seen an increasing number of publications in the last few years, especially after robust word embeddings models became prominent, when they proved themselves able to capture and represent semantic relationships from massive amounts of data. Nevertheless, traditional models often fall short in intrinsic issues of linguistics, such as polysemy and homonymy. Any expert system that makes use of natural language in its core, can be affected by a weak semantic representation of text, resulting in inaccurate outcomes based on poor decisions. To mitigate such issues, we propose a novel approach called Most Suitable Sense Annotation (MSSA), that disambiguates and annotates each word by its specific sense, considering the semantic effects of its context. Our approach brings three main contributions to the semantic representation scenario: (i) an unsupervised technique that disambiguates and annotates words by their senses, (ii) a multi-sense embeddings model that can be extended to any traditional word embeddings algorithm, and (iii) a recurrent methodology that allows our models to be re-used and their representations refined. We test our approach on six different benchmarks for the word similarity task, showing that our approach can produce state-of-the-art results and outperforms several more complex state-of-the-art systems.
An empirical evaluation of active inference in multi-armed bandits
Authors: Dimitrije Markovic, Hrvoje Stojic, Sarah Schwoebel, Stefan J. Kiebel
Date: 2021-01-21 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Decision Making Under Uncertainty, Multi-Armed Bandits, Sequential Decision Making, Thompson Sampling

A key feature of sequential decision making under uncertainty is a need to balance between exploiting--choosing the best action according to the current knowledge, and exploring--obtaining information about values of other actions. The multi-armed bandit problem, a classical task that captures this trade-off, served as a vehicle in machine learning for developing bandit algorithms that proved to be useful in numerous industrial applications. The active inference framework, an approach to sequential decision making recently developed in neuroscience for understanding human and animal behaviour, is distinguished by its sophisticated strategy for resolving the exploration-exploitation trade-off. This makes active inference an exciting alternative to already established bandit algorithms. Here we derive an efficient and scalable approximate active inference algorithm and compare it to two state-of-the-art bandit algorithms: Bayesian upper confidence bound and optimistic Thompson sampling. This comparison is done on two types of bandit problems: a stationary and a dynamic switching bandit. Our empirical evaluation shows that the active inference algorithm does not produce efficient long-term behaviour in stationary bandits. However, in the more challenging switching bandit problem active inference performs substantially better than the two state-of-the-art bandit algorithms. The results open exciting venues for further research in theoretical and applied machine learning, as well as lend additional credibility to active inference as a general framework for studying human and animal behaviour.
Validating Label Consistency in NER Data Annotation
Authors: Qingkai Zeng, Mengxia Yu, Wenhao Yu, Tianwen Jiang, Meng Jiang
Date: 2021-01-21 00:00:00
Proceeding: EMNLP (Eval4NLP) 2021 11
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER

Data annotation plays a crucial role in ensuring your named entity recognition (NER) projects are trained with the right information to learn from. Producing the most accurate labels is a challenge due to the complexity involved with annotation. Label inconsistency between multiple subsets of data annotation (e.g., training set and test set, or multiple training subsets) is an indicator of label mistakes. In this work, we present an empirical method to explore the relationship between label (in-)consistency and NER model performance. It can be used to validate the label consistency (or catches the inconsistency) in multiple sets of NER data annotation. In experiments, our method identified the label inconsistency of test data in SCIERC and CoNLL03 datasets (with 26.7% and 5.4% label mistakes). It validated the consistency in the corrected version of both datasets.
Sum-Rate-Distortion Function for Indirect Multiterminal Source Coding in Federated Learning
Authors: Naifu Zhang, Meixia Tao, Jia Wang
Date: 2021-01-21 00:00:00
Tasks: Decoder, Federated Learning

One of the main focus in federated learning (FL) is the communication efficiency since a large number of participating edge devices send their updates to the edge server at each round of the model training. Existing works reconstruct each model update from edge devices and implicitly assume that the local model updates are independent over edge devices. In FL, however, the model update is an indirect multi-terminal source coding problem, also called as the CEO problem where each edge device cannot observe directly the gradient that is to be reconstructed at the decoder, but is rather provided only with a noisy version. The existing works do not leverage the redundancy in the information transmitted by different edges. This paper studies the rate region for the indirect multiterminal source coding problem in FL. The goal is to obtain the minimum achievable rate at a particular upper bound of gradient variance. We obtain the rate region for the quadratic vector Gaussian CEO problem under unbiased estimator and derive an explicit formula of the sum-rate-distortion function in the special case where gradient are identical over edge device and dimension. Finally, we analyse communication efficiency of convex Minibatched SGD and non-convex Minibatched SGD based on the sum-rate-distortion function, respectively.
Characterizing signal propagation to close the performance gap in unnormalized ResNets
Authors: Andrew Brock, Soham De, Samuel L. Smith
Date: 2021-01-21 00:00:00
Proceeding: ICLR 2021 1

Batch Normalization is a key component in almost all state-of-the-art image classifiers, but it also introduces practical challenges: it breaks the independence between training examples within a batch, can incur compute and memory overhead, and often results in unexpected bugs. Building on recent theoretical analyses of deep ResNets at initialization, we propose a simple set of analysis tools to characterize signal propagation on the forward pass, and leverage these tools to design highly performant ResNets without activation normalization layers. Crucial to our success is an adapted version of the recently proposed Weight Standardization. Our analysis tools show how this technique preserves the signal in networks with ReLU or Swish activation functions by ensuring that the per-channel activation means do not grow with depth. Across a range of FLOP budgets, our networks attain performance competitive with the state-of-the-art EfficientNets on ImageNet.
On $L^q$ Convergence of the Hamiltonian Monte Carlo
Authors: Soumyadip Ghosh, Yingdong Lu, Tomasz Nowicki
Date: 2021-01-21 00:00:00

We establish $L_q$ convergence for Hamiltonian Monte Carlo algorithms. More specifically, under mild conditions for the associated Hamiltonian motion, we show that the outputs of the algorithms converge (strongly for $2\le q<\infty$ and weakly for $1<q<2$) to the desired target distribution.
Overfitting for Fun and Profit: Instance-Adaptive Data Compression
Authors: Ties van Rozendaal, Iris A. M. Huijben, Taco S. Cohen
Date: 2021-01-21 00:00:00
Proceeding: ICLR 2021 1
Tasks: Data Compression, Image Compression, Quantization

Neural data compression has been shown to outperform classical methods in terms of $RD$ performance, with results still improving rapidly. At a high level, neural compression is based on an autoencoder that tries to reconstruct the input instance from a (quantized) latent representation, coupled with a prior that is used to losslessly compress these latents. Due to limitations on model capacity and imperfect optimization and generalization, such models will suboptimally compress test data in general. However, one of the great strengths of learned compression is that if the test-time data distribution is known and relatively low-entropy (e.g. a camera watching a static scene, a dash cam in an autonomous car, etc.), the model can easily be finetuned or adapted to this distribution, leading to improved $RD$ performance. In this paper we take this concept to the extreme, adapting the full model to a single video, and sending model updates (quantized and compressed using a parameter-space prior) along with the latent representation. Unlike previous work, we finetune not only the encoder/latents but the entire model, and - during finetuning - take into account both the effect of model quantization and the additional costs incurred by sending the model updates. We evaluate an image compression model on I-frames (sampled at 2 fps) from videos of the Xiph dataset, and demonstrate that full-model adaptation improves $RD$ performance by ~1 dB, with respect to encoder-only finetuning.
ItNet: iterative neural networks with small graphs for accurate, efficient and anytime semantic segmentation
Authors: Thomas Pfeil
Date: 2021-01-21 00:00:00
Tasks: Semantic Segmentation

Deep neural networks have usually to be compressed and accelerated for their usage in low-power, e.g. mobile, devices. Recently, massively-parallel hardware accelerators were developed that offer high throughput and low latency at low power by utilizing in-memory computation. However, to exploit these benefits the computational graph of a neural network has to fit into the in-computation memory of these hardware systems that is usually rather limited in size. In this study, we introduce a class of network models that have a small memory footprint in terms of their computational graphs. To this end, the graph is designed to contain loops by iteratively executing a single network building block. Furthermore, the trade-off between accuracy and latency of these so-called iterative neural networks is improved by adding multiple intermediate outputs during both training and inference. We show state-of-the-art results for semantic segmentation on the CamVid and Cityscapes datasets that are especially demanding in terms of computational resources. In ablation studies, the improvement of network training by intermediate network outputs as well as the trade-off between weight sharing over iterations and the network size are investigated.
A two-stage data association approach for 3D Multi-object Tracking
Authors: Minh-Quan Dao, Vincent Frémont
Date: 2021-01-21 00:00:00
Tasks: 3D Multi-Object Tracking, 3D Object Detection, Autonomous Driving, Multi-Object Tracking, Object, object-detection, Object Detection, Object Tracking, Vocal Bursts Valence Prediction

Multi-object tracking (MOT) is an integral part of any autonomous driving pipelines because itproduces trajectories which has been taken by other moving objects in the scene and helps predicttheir future motion. Thanks to the recent advances in 3D object detection enabled by deep learning,track-by-detection has become the dominant paradigm in 3D MOT. In this paradigm, a MOT systemis essentially made of an object detector and a data association algorithm which establishes track-to-detection correspondence. While 3D object detection has been actively researched, associationalgorithms for 3D MOT seem to settle at a bipartie matching formulated as a linear assignmentproblem (LAP) and solved by the Hungarian algorithm. In this paper, we adapt a two-stage dataassociation method which was successful in image-based tracking to the 3D setting, thus providingan alternative for data association for 3D MOT. Our method outperforms the baseline using one-stagebipartie matching for data association by achieving 0.587 AMOTA in NuScenes validation set.
Estimating effective infection fatality rates during the course of the COVID-19 pandemic in Germany
Authors: Christian Staerk, Tobias Wistuba, Andreas Mayr
Date: 2020-11-04 00:00:00

The infection fatality rate (IFR) of the Coronavirus Disease 2019 (COVID-19) is one of the most discussed figures in the context of this pandemic. Using German COVID-19 surveillance data and age-group specific IFR estimates from multiple international studies, this work investigates time-dependent variations in effective IFR over the course of the pandemic. Three different methods for estimating (effective) IFRs are presented: (a) population-averaged IFRs based on the assumption that the infection risk is independent of age and time, (b) effective IFRs based on the assumption that the age distribution of confirmed cases approximately reflects the age distribution of infected individuals, and (c) effective IFRs accounting for age- and time-dependent dark figures of infections. Results show that effective IFRs in Germany are estimated to vary over time, as the age distributions of confirmed cases and estimated infections are changing during the course of the pandemic. In particular during the first and second waves of infections in spring and autumn/winter 2020, there has been a pronounced shift in the age distribution of confirmed cases towards older age groups, resulting in larger effective IFR estimates. The temporary increase in effective IFR during the first wave is estimated to be smaller but still remains when adjusting for age- and time-dependent dark figures. A comparison of effective IFRs with observed CFRs indicates that a substantial fraction of the time-dependent variability in observed mortality can be explained by changes in the age distribution of infections. Furthermore, a vanishing gap between effective IFRs and observed CFRs is apparent after the first infection wave, while a moderately increasing gap can be observed during the second wave. Further research is warranted to obtain timely age-stratified IFR estimates.
Regularization via deep generative models: an analysis point of view
Authors: Thomas Oberlin, Mathieu Verm
Date: 2021-01-21 00:00:00
Tasks: Deblurring, Decoder, Super-Resolution

This paper proposes a new way of regularizing an inverse problem in imaging (e.g., deblurring or inpainting) by means of a deep generative neural network. Compared to end-to-end models, such approaches seem particularly interesting since the same network can be used for many different problems and experimental conditions, as soon as the generative model is suited to the data. Previous works proposed to use a synthesis framework, where the estimation is performed on the latent vector, the solution being obtained afterwards via the decoder. Instead, we propose an analysis formulation where we directly optimize the image itself and penalize the latent vector. We illustrate the interest of such a formulation by running experiments of inpainting, deblurring and super-resolution. In many cases our technique achieves a clear improvement of the performance and seems to be more robust, in particular with respect to initialization.
A generalized ride-matching approach for sustainable shared mobility
Authors: Seyed Mehdi Meshkani, Bilal Farooq
Date: 2021-01-21 00:00:00

On-demand shared mobility is a promising and sustainable transportation approach that can mitigate vehicle externalities, such as traffic congestion and emission. On-demand shared mobility systems require matching of one (one-to-one) or multiple riders (many-to-one) to a vehicle based on real-time information. We propose a novel Graph-based Many-to-One ride-Matching (GMOMatch) algorithm for the dynamic many-to-one matching problem in the presence of traffic congestion. GMOMatch, which is an iterative two-step method, provides high service quality and is efficient in terms of computational complexity. It starts with a one-to-one matching in Step 1 and is followed by solving a maximum weight matching problem in Step 2 to combine the travel requests. To evaluate the performance, it is compared with a ride-matching algorithm developed by Simonetto et al. (2019). Both algorithms are implemented in a micro-traffic simulator to assess their performance and their impact on traffic congestion in Downtown, Toronto road network. In comparison to the Simonetto, GMOMatch improved the service rate, vehicle kilometer traveled and traffic travel time by 32%, 16.07%, and 4%, respectively. The sensitivity analysis indicated that utilizing vehicles with a capacity of 10 can achieve 25% service rate improvement compared to a capacity of 4.
Geometric Moment Invariants to Motion Blur
Authors: Hongxiang Hao., Hanlin Mo., Hua Li
Date: 2021-01-21 00:00:00
Tasks: Image Retrieval, Retrieval, Template Matching

In this paper, we focus on removing interference of motion blur by the derivation of motion blur invariants.Unlike earlier work, we don't restore any blurred image. Based on geometric moment and mathematical model of motion blur, we prove that geometric moments of blurred image and original image are linearly related. Depending on this property, we can analyse whether an existing moment-based feature is invariant to motion blur. Surprisingly, we find some geometric moment invariants are invariants to not only spatial transform but also motion blur. Meanwhile, we test invariance and robustness of these invariants using synthetic and real blur image datasets. And the results show these invariants outperform some widely used blur moment invariants and non-moment image features in image retrieval, classification and template matching.
Image-to-Image Translation: Methods and Applications
Authors: Yingxue Pang, Jianxin Lin, Tao Qin, Zhibo Chen
Date: 2021-01-21 00:00:00
Tasks: Image Generation, Image-to-Image Translation, Pose Estimation, Style Transfer, Translation

Image-to-image translation (I2I) aims to transfer images from a source domain to a target domain while preserving the content representations. I2I has drawn increasing attention and made tremendous progress in recent years because of its wide range of applications in many computer vision and image processing problems, such as image synthesis, segmentation, style transfer, restoration, and pose estimation. In this paper, we provide an overview of the I2I works developed in recent years. We will analyze the key techniques of the existing I2I works and clarify the main progress the community has made. Additionally, we will elaborate on the effect of I2I on the research and industry community and point out remaining challenges in related fields.
Noisy-target Training: A Training Strategy for DNN-based Speech Enhancement without Clean Speech
Authors: Takuya Fujimura, Yuma Koizumi, Kohei Yatabe, Ryoichi Miyazaki
Date: 2021-01-21 00:00:00
Tasks: Speech Enhancement, speech-recognition, Speech Recognition

Deep neural network (DNN)-based speech enhancement ordinarily requires clean speech signals as the training target. However, collecting clean signals is very costly because they must be recorded in a studio. This requirement currently restricts the amount of training data for speech enhancement to less than 1/1000 of that of speech recognition which does not need clean signals. Increasing the amount of training data is important for improving the performance, and hence the requirement of clean signals should be relaxed. In this paper, we propose a training strategy that does not require clean signals. The proposed method only utilizes noisy signals for training, which enables us to use a variety of speech signals in the wild. Our experimental results showed that the proposed method can achieve the performance similar to that of a DNN trained with clean signals.
Synwalk -- Community Detection via Random Walk Modelling
Authors: Christian Toth, Denis Helic, Bernhard C. Geiger
Date: 2021-01-21 00:00:00
Tasks: Community Detection

Complex systems, abstractly represented as networks, are ubiquitous in everyday life. Analyzing and understanding these systems requires, among others, tools for community detection. As no single best community detection algorithm can exist, robustness across a wide variety of problem settings is desirable. In this work, we present Synwalk, a random walk-based community detection method. Synwalk builds upon a solid theoretical basis and detects communities by synthesizing the random walk induced by the given network from a class of candidate random walks. We thoroughly validate the effectiveness of our approach on synthetic and empirical networks, respectively, and compare Synwalk's performance with the performance of Infomap and Walktrap. Our results indicate that Synwalk performs robustly on networks with varying mixing parameters and degree distributions. We outperform Infomap on networks with high mixing parameter, and Infomap and Walktrap on networks with many small communities and low average degree. Our work has a potential to inspire further development of community detection via synthesis of random walks and we provide concrete ideas for future research.
Mindless Attractor: A False-Positive Resistant Intervention for Drawing Attention Using Auditory Perturbation
Authors: Riku Arakawa, Hiromu Yakura
Date: 2021-01-21 00:00:00

Explicitly alerting users is not always an optimal intervention, especially when they are not motivated to obey. For example, in video-based learning, learners who are distracted from the video would not follow an alert asking them to pay attention. Inspired by the concept of Mindless Computing, we propose a novel intervention approach, Mindless Attractor, that leverages the nature of human speech communication to help learners refocus their attention without relying on their motivation. Specifically, it perturbs the voice in the video to direct their attention without consuming their conscious awareness. Our experiments not only confirmed the validity of the proposed approach but also emphasized its advantages in combination with a machine learning-based sensing module. Namely, it would not frustrate users even though the intervention is activated by false-positive detection of their attentive state. Our intervention approach can be a reliable way to induce behavioral change in human-AI symbiosis.