pwc-archive/papers-with-abstracts huggingface dataset:
CTC-synchronous Training for Monotonic Attention Model
Authors: Hirofumi Inaguma, Masato Mimura, Tatsuya Kawahara
Date: 2020-05-10 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Decoder, model, speech-recognition, Speech Recognition

Monotonic chunkwise attention (MoChA) has been studied for the online streaming automatic speech recognition (ASR) based on a sequence-to-sequence framework. In contrast to connectionist temporal classification (CTC), backward probabilities cannot be leveraged in the alignment marginalization process during training due to left-to-right dependency in the decoder. This results in the error propagation of alignments to subsequent token generation. To address this problem, we propose CTC-synchronous training (CTC-ST), in which MoChA uses CTC alignments to learn optimal monotonic alignments. Reference CTC alignments are extracted from a CTC branch sharing the same encoder with the decoder. The entire model is jointly optimized so that the expected boundaries from MoChA are synchronized with the alignments. Experimental evaluations of the TEDLIUM release-2 and Librispeech corpora show that the proposed method significantly improves recognition, especially for long utterances. We also show that CTC-ST can bring out the full potential of SpecAugment for MoChA.
A Reinforcement Learning based approach for Multi-target Detection in Massive MIMO radar
Authors: Aya Mostafa Ahmed, Alaa Alameer Ahmad, Stefano Fortunati, Aydin Sezgin, Maria S. Greco, Fulvio Gini
Date: 2020-05-10 00:00:00
Tasks: reinforcement-learning, Reinforcement Learning (RL)

This paper considers the problem of multi-target detection for massive multiple input multiple output (MMIMO) cognitive radar (CR). The concept of CR is based on the perception-action cycle that senses and intelligently adapts to the dynamic environment in order to optimally satisfy a specific mission. However, this usually requires a priori knowledge of the environmental model, which is not available in most cases. We propose a reinforcement learning (RL) based algorithm for cognitive multi-target detection in the presence of unknown disturbance statistics. The radar acts as an agent that continuously senses the unknown environment (i.e., targets and disturbance) and consequently optimizes transmitted waveforms in order to maximize the probability of detection ($P_\mathsf{D}$) by focusing the energy in specific range-angle cells (i.e., beamforming). Furthermore, we propose a solution to the beamforming optimization problem with less complexity than the existing methods. Numerical simulations are performed to assess the performance of the proposed RL-based algorithm in both stationary and dynamic environments. The RL based beamforming is compared to the conventional omnidirectional approach with equal power allocation and to adaptive beamforming with no RL. As highlighted by the proposed numerical results, our RL-based beamformer outperforms both approaches in terms of target detection performance. The performance improvement is even particularly remarkable under environmentally harsh conditions such as low SNR, heavy-tailed disturbance and rapidly changing scenarios.
Hierarchical Regression Network for Spectral Reconstruction from RGB Images
Authors: Yuzhi Zhao, Lai-Man Po, Qiong Yan, Wei Liu, Tingyu Lin
Date: 2020-05-10 00:00:00
Tasks: Decoder, regression, Spectral Reconstruction

Capturing visual image with a hyperspectral camera has been successfully applied to many areas due to its narrow-band imaging technology. Hyperspectral reconstruction from RGB images denotes a reverse process of hyperspectral imaging by discovering an inverse response function. Current works mainly map RGB images directly to corresponding spectrum but do not consider context information explicitly. Moreover, the use of encoder-decoder pair in current algorithms leads to loss of information. To address these problems, we propose a 4-level Hierarchical Regression Network (HRNet) with PixelShuffle layer as inter-level interaction. Furthermore, we adopt a residual dense block to remove artifacts of real world RGB images and a residual global block to build attention mechanism for enlarging perceptive field. We evaluate proposed HRNet with other architectures and techniques by participating in NTIRE 2020 Challenge on Spectral Reconstruction from RGB Images. The HRNet is the winning method of track 2 - real world images and ranks 3rd on track 1 - clean images. Please visit the project web page https://github.com/zhaoyuzhi/Hierarchical-Regression-Network-for-Spectral-Reconstruction-from-RGB-Images to try our codes and pre-trained models.
Segmentation of Macular Edema Datasets with Small Residual 3D U-Net Architectures
Authors: Jonathan Frawley, Chris G. Willcocks, Maged Habib, Caspar Geenen, David H. Steel, Boguslaw Obara
Date: 2020-05-10 00:00:00

This paper investigates the application of deep convolutional neural networks with prohibitively small datasets to the problem of macular edema segmentation. In particular, we investigate several different heavily regularized architectures. We find that, contrary to popular belief, neural architectures within this application setting are able to achieve close to human-level performance on unseen test images without requiring large numbers of training examples. Annotating these 3D datasets is difficult, with multiple criteria required. It takes an experienced clinician two days to annotate a single 3D image, whereas our trained model achieves similar performance in less than a second. We found that an approach which uses targeted dataset augmentation, alongside architectural simplification with an emphasis on residual design, has acceptable generalization performance - despite relying on fewer than 15 training examples.
Topological regularization with information filtering networks
Authors: Tomaso Aste
Date: 2020-05-10 00:00:00
Tasks: regression

A methodology to perform topological regularization via information filtering network is introduced. This methodology can be directly applied to covariance selection problem providing an instrument for sparse probabilistic modeling with both linear and non-linear multivariate probability distributions such as the elliptical and generalized hyperbolic families. It can also be directly implemented for $L_0$-norm regularized multicollinear regression. In this paper, I describe in detail an application to sparse modeling with multivariate Student-t. A specific $L_0$-norm regularized expectation-maximization likelihood maximization procedure is proposed for this sparse Student-t case. Examples with real data from stock prices log-returns and from artificially generated data demonstrate the applicability, performances, and potentials of this methodology.
Improving The Performance Of The K-means Algorithm
Authors: Tien-Dung Nguyen
Date: 2020-05-10 00:00:00
Tasks: Clustering

The Incremental K-means (IKM), an improved version of K-means (KM), was introduced to improve the clustering quality of KM significantly. However, the speed of IKM is slower than KM. My thesis proposes two algorithms to speed up IKM while remaining the quality of its clustering result approximately. The first algorithm, called Divisive K-means, improves the speed of IKM by speeding up its splitting process of clusters. Testing with UCI Machine Learning data sets, the new algorithm achieves the empirically global optimum as IKM and has lower complexity, $O(k*log_{2}k*n)$, than IKM, $O(k^{2}n)$. The second algorithm, called Parallel Two-Phase K-means (Par2PK-means), parallelizes IKM by employing the model of Two-Phase K-means. Testing with large data sets, this algorithm attains a good speedup ratio, closing to the linearly speed-up ratio.
Non-Autoregressive Image Captioning with Counterfactuals-Critical Multi-Agent Learning
Authors: Longteng Guo, Jing Liu, Xinxin Zhu, Xingjian He, Jie Jiang, Hanqing Lu
Date: 2020-05-10 00:00:00
Tasks: Image Captioning, Machine Translation, Multi-agent Reinforcement Learning, Sentence, Translation

Most image captioning models are autoregressive, i.e. they generate each word by conditioning on previously generated words, which leads to heavy latency during inference. Recently, non-autoregressive decoding has been proposed in machine translation to speed up the inference time by generating all words in parallel. Typically, these models use the word-level cross-entropy loss to optimize each word independently. However, such a learning process fails to consider the sentence-level consistency, thus resulting in inferior generation quality of these non-autoregressive models. In this paper, we propose a Non-Autoregressive Image Captioning (NAIC) model with a novel training paradigm: Counterfactuals-critical Multi-Agent Learning (CMAL). CMAL formulates NAIC as a multi-agent reinforcement learning system where positions in the target sequence are viewed as agents that learn to cooperatively maximize a sentence-level reward. Besides, we propose to utilize massive unlabeled images to boost captioning performance. Extensive experiments on MSCOCO image captioning benchmark show that our NAIC model achieves a performance comparable to state-of-the-art autoregressive models, while brings 13.9x decoding speedup.
From Standard Summarization to New Tasks and Beyond: Summarization with Manifold Information
Authors: Shen Gao, Xiuying Chen, Zhaochun Ren, Dongyan Zhao, Rui Yan
Date: 2020-05-10 00:00:00
Tasks: Text Summarization

Text summarization is the research area aiming at creating a short and condensed version of the original document, which conveys the main idea of the document in a few words. This research topic has started to attract the attention of a large community of researchers, and it is nowadays counted as one of the most promising research areas. In general, text summarization algorithms aim at using a plain text document as input and then output a summary. However, in real-world applications, most of the data is not in a plain text format. Instead, there is much manifold information to be summarized, such as the summary for a web page based on a query in the search engine, extreme long document (e.g., academic paper), dialog history and so on. In this paper, we focus on the survey of these new summarization tasks and approaches in the real-world application.
Optimizing Deep Learning Recommender Systems' Training On CPU Cluster Architectures
Authors: Dhiraj Kalamkar, Evangelos Georganas, Sudarshan Srinivasan, Jianping Chen, Mikhail Shiryaev, Alexander Heinecke
Date: 2020-05-10 00:00:00
Tasks: Cloud Computing, CPU, Deep Learning, Recommendation Systems

During the last two years, the goal of many researchers has been to squeeze the last bit of performance out of HPC system for AI tasks. Often this discussion is held in the context of how fast ResNet50 can be trained. Unfortunately, ResNet50 is no longer a representative workload in 2020. Thus, we focus on Recommender Systems which account for most of the AI cycles in cloud computing centers. More specifically, we focus on Facebook's DLRM benchmark. By enabling it to run on latest CPU hardware and software tailored for HPC, we are able to achieve more than two-orders of magnitude improvement in performance (110x) on a single socket compared to the reference CPU implementation, and high scaling efficiency up to 64 sockets, while fitting ultra-large datasets. This paper discusses the optimization techniques for the various operators in DLRM and which component of the systems are stressed by these different operators. The presented techniques are applicable to a broader set of DL workloads that pose the same scaling challenges/characteristics as DLRM.
HNet: Graphical Hypergeometric Networks
Authors: Erdogan Taskesen
Date: 2020-05-10 00:00:00

Motivation: Real-world data often contain measurements with both continuous and discrete values. Despite the availability of many libraries, data sets with mixed data types require intensive pre-processing steps, and it remains a challenge to describe the relationships between variables. The data understanding phase is an important step in the data mining process, however, without making any assumptions on the data, the search space is super-exponential in the number of variables. Methods: We propose graphical hypergeometric networks (HNet), a method to test associations across variables for significance using statistical inference. The aim is to determine a network using only the significant associations in order to shed light on the complex relationships across variables. HNet processes raw unstructured data sets and outputs a network that consists of (partially) directed or undirected edges between the nodes (i.e., variables). To evaluate the accuracy of HNet, we used well known data sets and in addition generated data sets with known ground truth. The performance of HNet is compared to Bayesian structure learning. Results: We demonstrate that HNet showed high accuracy and performance in the detection of node links. In the case of the Alarm data set we can demonstrate on average an MCC score of 0.33 + 0.0002 (P<1x10-6), whereas Bayesian structure learning resulted in an average MCC score of 0.52 + 0.006 (P<1x10-11), and randomly assigning edges resulted in a MCC score of 0.004 + 0.0003 (P=0.49). Conclusions: HNet can process raw unstructured data sets, allows analysis of mixed data types, it easily scales up in number of variables, and allows detailed examination of the detected associations. Availability: https://erdogant.github.io/hnet/
A Generalized Kernel Risk Sensitive Loss for Robust Two-Dimensional Singular Value Decomposition
Authors: Miaohua Zhang, Yongsheng Gao
Date: 2020-05-10 00:00:00
Tasks: Clustering, Image Reconstruction

Two-dimensional singular decomposition (2DSVD) has been widely used for image processing tasks, such as image reconstruction, classification, and clustering. However, traditional 2DSVD algorithm is based on the mean square error (MSE) loss, which is sensitive to outliers. To overcome this problem, we propose a robust 2DSVD framework based on a generalized kernel risk sensitive loss (GKRSL-2DSVD) which is more robust to noise and and outliers. Since the proposed objective function is non-convex, a majorization-minimization algorithm is developed to efficiently solve it with guaranteed convergence. The proposed framework has inherent properties of processing non-centered data, rotational invariant, being easily extended to higher order spaces. Experimental results on public databases demonstrate that the performance of the proposed method on different applications significantly outperforms that of all the benchmarks.
Search In Power-Law Networks
Authors: Lada Adamic, Rajan Lukose, Amit Puniyani, Bernardo Huberman
Date: 2020-05-13 00:00:00
Proceeding: Physica A 2020 5

Many communication and social networks have power-law link distributions, containing a few nodes which have a very high degree and many with low degree. The high connectivity nodes play the important role of hubs in communication and networking, a fact which can be exploited when designing efficient search algorithms. We introduce a number of local search strategies which utilize high degree nodes in power-law graphs and which have costs which scale sub-linearly with the size of the graph. We also demonstrate the utility of these strategies on the Gnutella peer-to-peer network.
SubNets of Scale-Free Networks Are Not Scale-Free: Sampling Properties of Networks
Authors: Michael P. H. Stumpf, Carsten Wiuf, Robert M. May
Date: 2020-05-13 00:00:00
Proceeding: PNAS 2020 5

Most studies of networks have only looked at small subsets of the true network. Here, we discuss the sampling properties of a network's degree distribution under the most parsimonious sampling scheme. Only if the degree distributions of the network and randomly sampled subnets belong to the same family of probability distributions is it possible to extrapolate from subnet data to properties of the global network. We show that this condition is indeed satisfied for some important classes of networks, notably classical random graphs and exponential random graphs. For scale-free degree distributions, however, this is not the case. Thus, inferences about the scale-free nature of a network may have to be treated with some caution. The work presented here has important implications for the analysis of molecular networks as well as for graph theory and the theory of networks in general.
Epipolar Transformers
Authors: Yihui He, Rui Yan, Katerina Fragkiadaki, Shoou-I Yu
Date: 2020-05-10 00:00:00
Proceeding: CVPR 2020 6
Tasks: 2D Pose Estimation, 3D Hand Pose Estimation, 3D Human Pose Estimation, 3D Pose Estimation, Pose Estimation, Stereo Matching

A common approach to localize 3D human joints in a synchronized and calibrated multi-view setup consists of two-steps: (1) apply a 2D detector separately on each view to localize joints in 2D, and (2) perform robust triangulation on 2D detections from each view to acquire the 3D joint locations. However, in step 1, the 2D detector is limited to solving challenging cases which could potentially be better resolved in 3D, such as occlusions and oblique viewing angles, purely in 2D without leveraging any 3D information. Therefore, we propose the differentiable "epipolar transformer", which enables the 2D detector to leverage 3D-aware features to improve 2D pose estimation. The intuition is: given a 2D location p in the current view, we would like to first find its corresponding point p' in a neighboring view, and then combine the features at p' with the features at p, thus leading to a 3D-aware feature at p. Inspired by stereo matching, the epipolar transformer leverages epipolar constraints and feature matching to approximate the features at p'. Experiments on InterHand and Human3.6M show that our approach has consistent improvements over the baselines. Specifically, in the condition where no external data is used, our Human3.6M model trained with ResNet-50 backbone and image size 256 x 256 outperforms state-of-the-art by 4.23 mm and achieves MPJPE 26.9 mm.
Sampling From Large Graphs
Authors: Jure Leskovec, Christos Faloutsos
Date: 2006-08-21 00:00:00
Proceeding: KDD 2006 8
Tasks: Graph Sampling, Natural Questions

Given a huge real graph, how can we derive a representative sample? There are many known algorithms to compute interesting measures (shortest paths, centrality, betweenness, etc.), but several of them become impractical for large graphs. Thus graph sampling is essential.The natural questions to ask are (a) which sampling method to use, (b) how small can the sample size be, and (c) how to scale up the measurements of the sample (e.g., the diameter), to get estimates for the large graph. The deeper, underlying question is subtle: how do we measure success?.We answer the above questions, and test our answers by thorough experiments on several, diverse datasets, spanning thousands nodes and edges. We consider several sampling methods, propose novel methods to check the goodness of sampling, and develop a set of scaling laws that describe relations between the properties of the original and the sample.In addition to the theoretical contributions, the practical conclusions from our work are: Sampling strategies based on edge selection do not perform well; simple uniform random node selection performs surprisingly well. Overall, best performing methods are the ones based on random-walks and "forest fire"; they match very accurately both static as well as evolutionary graph patterns, with sample sizes down to about 15% of the original graph.
Reducing Large Internet Topologies for Faster Simulations
Authors: V. Krishnamurthy, M. Faloutsos, M. Chrobak, L. Lao, J. -H. Cui, A. G. Percus
Date: 2020-05-13 00:00:00
Proceeding: ‏‏‎ ‎ 2020 5
Tasks: Graph Sampling

In this paper, we develop methods to “sample” a small realistic graph from a large real network. Despite recent activity, the modeling and generation of realistic graphs is still not a resolved issue. All previous work has attempted to grow a graph from scratch. We address the complementary problem of shrinking a graph. In more detail, this work has three parts. First, we propose a number of reduction methods that can be categorized into three classes: (a) deletion methods, (b) contraction methods, and (c) exploration methods. We prove that some of them maintain key properties of the initial graph. We implement our methods and show that we can effectively reduce the nodes of a graph by as much as 70% while maintaining its important properties. In addition, we show that our reduced graphs compare favourably against construction-based generators. Apart from its use in simulations, the problem of graph sampling is of independent interest.
Network Sampling: From Static to Streaming Graphs
Authors: Nesreen K. Ahmed, Jennifer L Neville, Ramana Rao
Date: 2020-05-13 00:00:00
Proceeding: ‏‏‎ ‎ 2020 5
Tasks: parameter estimation

Network sampling is integral to the analysis of social, information, and biological networks. Since many real-world networks are massive in size, continuously evolving, and/or distributed in nature, the network structure is often sampled in order to facilitate study. For these reasons, a more thorough and complete understanding of network sampling is critical to support the field of network science. In this paper, we outline a framework for the general problem of network sampling by highlighting the different objectives, population and units of interest, and classes of network sampling methods. In addition, we propose a spectrum of computational models for network sampling methods, ranging from the traditionally studied model based on the assumption of a static domain to a more challenging model that is appropriate for streaming domains. We design a family of sampling methods based on the concept of graph induction that generalize across the full spectrum of computational models (from static to streaming) while efficiently preserving many of the topological properties of the input graphs. Furthermore, we demonstrate how traditional static sampling algorithms can be modified for graph streams for each of the three main classes of sampling methods: node, edge, and topology-based sampling. Experimental results indicate that our proposed family of sampling methods more accurately preserve the underlying properties of the graph in both static and streaming domains. Finally, we study the impact of network sampling algorithms on the parameter estimation and performance evaluation of relational classification algorithms.
Benchmarking Flexible Electric Loads Scheduling Algorithms under Market Price Uncertainty
Authors: Koos van der Linden, Natalia Romero, Mathijs M. de Weerdt
Date: 2020-02-04 00:00:00
Tasks: Benchmarking, Decision Making, Scheduling

Because of increasing amounts of intermittent and distributed generators in power systems, many demand response programs have been developed to schedule flexible energy consumption. However, proper benchmarks for comparing these methods are lacking, especially when decisions are made based on information which is repeatedly updated. This paper presents a new benchmarking tool designed to bridge this gap. Surveys that classify flexibility planning algorithms were an input to define the benchmarking standards. The framework can be used for different objectives and under diverse conditions faced by electricity energy stakeholders interested in flexibility scheduling algorithms. It includes a simulation environment that captures the evolution of look--ahead information, which enables comparing online planning and scheduling algorithms. The benchmarking tool is used to test seven planning algorithms measuring their performance under uncertain market conditions. The results show the importance of online decision making, the influence of data quality on the algorithm performance, the benefit of using robust and stochastic programming approaches, and the necessity of trustworthy benchmarking.