pwc-archive/papers-with-abstracts huggingface dataset:
Time-series Generative Adversarial Networks
Authors: Jinsung Yoon, Daniel Jarrett, Mihaela van der Schaar
Date: 2019-12-01 00:00:00
Tasks: Time Series, Time Series Analysis

A good generative model for time-series data should preservetemporal dynamics,in the sense that new sequences respect the original relationships between variablesacross time. Existing methods that bring generative adversarial networks (GANs)into the sequential setting do not adequately attend to the temporal correlationsunique to time-series data.  At the same time, supervised models for sequenceprediction—which allow finer control over network dynamics—are inherentlydeterministic. We propose a novel framework for generating realistic time-seriesdata that combines the flexibility of the unsupervised paradigm with the controlafforded  by  supervised  training.   Through  a  learned  embedding  space  jointlyoptimized  with  both  supervised  and  adversarial  objectives,  we  encourage  thenetwork to adhere to the dynamics of the training data during sampling. Empirically,we evaluate the ability of our method to generate realistic samples using a variety ofreal and synthetic time-series datasets. Qualitatively and quantitatively, we find thatthe proposed framework consistently and significantly outperforms state-of-the-artbenchmarks with respect to measures of similarity and predictive ability.
Extended Full Block S-Procedure for Distributed Control of Interconnected Systems
Authors: Giulia De Pasquale, Yvonne R. Sturz, Maria Elena Valcher, Roy S. Smith
Date: 2020-12-11 00:00:00

This paper proposes a novel method for distributed controller synthesis of homogeneous interconnected systems consisting of identical subsystems. The objective of the designed controller is to minimize the L2-gain of the performance channel. The proposed method is an extended formulation of the Full Block S-Procedure (FBSP) where we introduce an additional set of variables. This allows relaxing the block-diagonal structural assumptions on the Lyapunov and multiplier matrices required for distributed control design, which reduces conservatism w.r.t most existing approaches. We show how to decompose the proposed extended FBSP into small synthesis conditions, of the size of one individual subsystem.
DEAAN: Disentangled Embedding and Adversarial Adaptation Network for Robust Speaker Representation Learning
Authors: Mufan Sang, Wei Xia, John H. L. Hansen
Date: 2020-12-12 00:00:00
Tasks: Disentanglement, Domain Adaptation, Representation Learning, Speaker Verification

Despite speaker verification has achieved significant performance improvement with the development of deep neural networks, domain mismatch is still a challenging problem in this field. In this study, we propose a novel framework to disentangle speaker-related and domain-specific features and apply domain adaptation on the speaker-related feature space solely. Instead of performing domain adaptation directly on the feature space where domain information is not removed, using disentanglement can efficiently boost adaptation performance. To be specific, our model's input speech from the source and target domains is first encoded into different latent feature spaces. The adversarial domain adaptation is conducted on the shared speaker-related feature space to encourage the property of domain-invariance. Further, we minimize the mutual information between speaker-related and domain-specific features for both domains to enforce the disentanglement. Experimental results on the VOiCES dataset demonstrate that our proposed framework can effectively generate more speaker-discriminative and domain-invariant speaker representations with a relative 20.3% reduction of EER compared to the original ResNet-based system.
Improving Image Captioning by Leveraging Intra- and Inter-layer Global Representation in Transformer Network
Authors: Jiayi Ji, Yunpeng Luo, Xiaoshuai Sun, Fuhai Chen, Gen Luo, Yongjian Wu, Yue Gao, Rongrong Ji
Date: 2020-12-13 00:00:00
Tasks: Caption Generation, Decoder, Image Captioning

Transformer-based architectures have shown great success in image captioning, where object regions are encoded and then attended into the vectorial representations to guide the caption decoding. However, such vectorial representations only contain region-level information without considering the global information reflecting the entire image, which fails to expand the capability of complex multi-modal reasoning in image captioning. In this paper, we introduce a Global Enhanced Transformer (termed GET) to enable the extraction of a more comprehensive global representation, and then adaptively guide the decoder to generate high-quality captions. In GET, a Global Enhanced Encoder is designed for the embedding of the global feature, and a Global Adaptive Decoder are designed for the guidance of the caption generation. The former models intra- and inter-layer global representation by taking advantage of the proposed Global Enhanced Attention and a layer-wise fusion module. The latter contains a Global Adaptive Controller that can adaptively fuse the global information into the decoder to guide the caption generation. Extensive experiments on MS COCO dataset demonstrate the superiority of our GET over many state-of-the-arts.
Informer: Beyond Efficient Transformer for Long Sequence Time-Series Forecasting
Authors: Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, JianXin Li, Hui Xiong, Wancai Zhang
Date: 2020-12-14 00:00:00
Tasks: Decoder, Multivariate Time Series Forecasting, Time Series, Time Series Analysis, Time Series Forecasting, Univariate Time Series Forecasting

Many real-world applications require the prediction of long sequence time-series, such as electricity consumption planning. Long sequence time-series forecasting (LSTF) demands a high prediction capacity of the model, which is the ability to capture precise long-range dependency coupling between output and input efficiently. Recent studies have shown the potential of Transformer to increase the prediction capacity. However, there are several severe issues with Transformer that prevent it from being directly applicable to LSTF, including quadratic time complexity, high memory usage, and inherent limitation of the encoder-decoder architecture. To address these issues, we design an efficient transformer-based model for LSTF, named Informer, with three distinctive characteristics: (i) a $ProbSparse$ self-attention mechanism, which achieves $O(L \log L)$ in time complexity and memory usage, and has comparable performance on sequences' dependency alignment. (ii) the self-attention distilling highlights dominating attention by halving cascading layer input, and efficiently handles extreme long input sequences. (iii) the generative style decoder, while conceptually simple, predicts the long time-series sequences at one forward operation rather than a step-by-step way, which drastically improves the inference speed of long-sequence predictions. Extensive experiments on four large-scale datasets demonstrate that Informer significantly outperforms existing methods and provides a new solution to the LSTF problem.
Efficient Incorporation of Multiple Latency Targets in the Once-For-All Network
Authors: Vidhur Kumar, Andrew Szidon
Date: 2020-12-12 00:00:00
Tasks: All, Network Pruning, Neural Architecture Search

Neural Architecture Search has proven an effective method of automating architecture engineering. Recent work in the field has been to look for architectures subject to multiple objectives such as accuracy and latency to efficiently deploy them on different target hardware. Once-for-All (OFA) is one such method that decouples training and search and is able to find high-performance networks for different latency constraints. However, the search phase is inefficient at incorporating multiple latency targets. In this paper, we introduce two strategies (Top-down and Bottom-up) that use warm starting and randomized network pruning for the efficient incorporation of multiple latency targets in the OFA network. We evaluate these strategies against the current OFA implementation and demonstrate that our strategies offer significant running time performance gains while not sacrificing the accuracy of the subnetworks that were found for each latency target. We further demonstrate that these performance gains are generalized to every design space used by the OFA network.
Learning Relaxed Belady for Content Distribution Network Caching
Authors: Zhenyu Song, Daniel S. Berger, Kai Li, Wyatt Lloyd
Date: 2020-02-25 00:00:00
Tasks: BIG-bench Machine Learning

This paper presents a new approach for caching in CDNs that uses machine learning to approximate the Belady MIN algorithm. To accomplish this complex task, we introduce the Relaxed Belady algorithm, the Belady boundary, and the good decision ratio that inform the design of Learning Relaxed Belady (LRB). LRB addresses the necessary system challenges to build an end-to-end machine learning caching prototype, including how to gather training data, limit memory overhead, and have lightweight training and inference paths.

We implement an LRB simulator and a prototype within Apache Traffic Server. Our simulation using 6 production CDN traces show LRB reduces WAN traffic compared to a typical production CDN cache design by 5–24%, and consistently outperform other state-of-the-art methods. Our evaluation of the LRB prototype shows its overhead is modest and it can be deployed on today’s CDN servers.
Accurate Cell Segmentation in Digital Pathology Images via Attention Enforced Networks
Authors: Muyi Sun, Zeyi Yao, Guanhong Zhang
Date: 2020-12-14 00:00:00
Tasks: Cell Segmentation, Color Normalization, Segmentation

Automatic cell segmentation is an essential step in the pipeline of computer-aided diagnosis (CAD), such as the detection and grading of breast cancer. Accurate segmentation of cells can not only assist the pathologists to make a more precise diagnosis, but also save much time and labor. However, this task suffers from stain variation, cell inhomogeneous intensities, background clutters and cells from different tissues. To address these issues, we propose an Attention Enforced Network (AENet), which is built on spatial attention module and channel attention module, to integrate local features with global dependencies and weight effective channels adaptively. Besides, we introduce a feature fusion branch to bridge high-level and low-level features. Finally, the marker controlled watershed algorithm is applied to post-process the predicted segmentation maps for reducing the fragmented regions. In the test stage, we present an individual color normalization method to deal with the stain variation problem. We evaluate this model on the MoNuSeg dataset. The quantitative comparisons against several prior methods demonstrate the superiority of our approach.
MSAF: Multimodal Split Attention Fusion
Authors: Lang Su, Chuqing Hu, Guofa Li, Dongpu Cao
Date: 2020-12-13 00:00:00
Tasks: Action Recognition, Emotion Recognition, Multimodal Emotion Recognition, Multimodal Sentiment Analysis, Sentiment Analysis

Multimodal learning mimics the reasoning process of the human multi-sensory system, which is used to perceive the surrounding world. While making a prediction, the human brain tends to relate crucial cues from multiple sources of information. In this work, we propose a novel multimodal fusion module that learns to emphasize more contributive features across all modalities. Specifically, the proposed Multimodal Split Attention Fusion (MSAF) module splits each modality into channel-wise equal feature blocks and creates a joint representation that is used to generate soft attention for each channel across the feature blocks. Further, the MSAF module is designed to be compatible with features of various spatial dimensions and sequence lengths, suitable for both CNNs and RNNs. Thus, MSAF can be easily added to fuse features of any unimodal networks and utilize existing pretrained unimodal model weights. To demonstrate the effectiveness of our fusion module, we design three multimodal networks with MSAF for emotion recognition, sentiment analysis, and action recognition tasks. Our approach achieves competitive results in each task and outperforms other application-specific networks and multimodal fusion benchmarks.
Efficient Human Pose Estimation by Learning Deeply Aggregated Representations
Authors: Zhengxiong Luo, Zhicheng Wang, Yuanhao Cai, GuanAn Wang, Yan Huang, Liang Wang, Erjin Zhou, Tieniu Tan, Jian Sun
Date: 2020-12-13 00:00:00
Tasks: CPU, Pose Estimation

In this paper, we propose an efficient human pose estimation network (DANet) by learning deeply aggregated representations. Most existing models explore multi-scale information mainly from features with different spatial sizes. Powerful multi-scale representations usually rely on the cascaded pyramid framework. This framework largely boosts the performance but in the meanwhile makes networks very deep and complex. Instead, we focus on exploiting multi-scale information from layers with different receptive-field sizes and then making full of use this information by improving the fusion method. Specifically, we propose an orthogonal attention block (OAB) and a second-order fusion unit (SFU). The OAB learns multi-scale information from different layers and enhances them by encouraging them to be diverse. The SFU adaptively selects and fuses diverse multi-scale information and suppress the redundant ones. This could maximize the effective information in final fused representations. With the help of OAB and SFU, our single pyramid network may be able to generate deeply aggregated representations that contain even richer multi-scale information and have a larger representing capacity than that of cascaded networks. Thus, our networks could achieve comparable or even better accuracy with much smaller model complexity. Specifically, our \mbox{DANet-72} achieves $70.5$ in AP score on COCO test-dev set with only $1.0G$ FLOPs. Its speed on a CPU platform achieves $58$ Persons-Per-Second~(PPS).
DFR: Deep Feature Reconstruction for Unsupervised Anomaly Segmentation
Authors: Jie Yang, Yong Shi, Zhiquan Qi
Date: 2020-12-13 00:00:00
Tasks: Anomaly Detection, Anomaly Segmentation, Descriptive

Automatic detecting anomalous regions in images of objects or textures without priors of the anomalies is challenging, especially when the anomalies appear in very small areas of the images, making difficult-to-detect visual variations, such as defects on manufacturing products. This paper proposes an effective unsupervised anomaly segmentation approach that can detect and segment out the anomalies in small and confined regions of images. Concretely, we develop a multi-scale regional feature generator that can generate multiple spatial context-aware representations from pre-trained deep convolutional networks for every subregion of an image. The regional representations not only describe the local characteristics of corresponding regions but also encode their multiple spatial context information, making them discriminative and very beneficial for anomaly detection. Leveraging these descriptive regional features, we then design a deep yet efficient convolutional autoencoder and detect anomalous regions within images via fast feature reconstruction. Our method is simple yet effective and efficient. It advances the state-of-the-art performances on several benchmark datasets and shows great potential for real applications.
Disentangled Information Bottleneck
Authors: Ziqi Pan, Li Niu, Jianfu Zhang, Liqing Zhang
Date: 2020-12-14 00:00:00
Tasks: Adversarial Attack, Out-of-Distribution Detection, Prediction

The information bottleneck (IB) method is a technique for extracting information that is relevant for predicting the target random variable from the source random variable, which is typically implemented by optimizing the IB Lagrangian that balances the compression and prediction terms. However, the IB Lagrangian is hard to optimize, and multiple trials for tuning values of Lagrangian multiplier are required. Moreover, we show that the prediction performance strictly decreases as the compression gets stronger during optimizing the IB Lagrangian. In this paper, we implement the IB method from the perspective of supervised disentangling. Specifically, we introduce Disentangled Information Bottleneck (DisenIB) that is consistent on compressing source maximally without target prediction performance loss (maximum compression). Theoretical and experimental results demonstrate that our method is consistent on maximum compression, and performs well in terms of generalization, robustness to adversarial attack, out-of-distribution detection, and supervised disentangling.
Self-Supervised Hypergraph Convolutional Networks for Session-based Recommendation
Authors: Xin Xia, Hongzhi Yin, Junliang Yu, Qinyong Wang, Lizhen Cui, Xiangliang Zhang
Date: 2020-12-12 00:00:00
Tasks: Self-Supervised Learning, Session-Based Recommendations

Session-based recommendation (SBR) focuses on next-item prediction at a certain time point. As user profiles are generally not available in this scenario, capturing the user intent lying in the item transitions plays a pivotal role. Recent graph neural networks (GNNs) based SBR methods regard the item transitions as pairwise relations, which neglect the complex high-order information among items. Hypergraph provides a natural way to capture beyond-pairwise relations, while its potential for SBR has remained unexplored. In this paper, we fill this gap by modeling session-based data as a hypergraph and then propose a hypergraph convolutional network to improve SBR. Moreover, to enhance hypergraph modeling, we devise another graph convolutional network which is based on the line graph of the hypergraph and then integrate self-supervised learning into the training of the networks by maximizing mutual information between the session representations learned via the two networks, serving as an auxiliary task to improve the recommendation task. Since the two types of networks both are based on hypergraph, which can be seen as two channels for hypergraph modeling, we name our model \textbf{DHCN} (Dual Channel Hypergraph Convolutional Networks). Extensive experiments on three benchmark datasets demonstrate the superiority of our model over the SOTA methods, and the results validate the effectiveness of hypergraph modeling and self-supervised task. The implementation of our model is available at https://github.com/xiaxin1998/DHCN
Epistemic Logic of Know-Who
Authors: Sophia Epstein, Pavel Naumov
Date: 2020-12-11 00:00:00

The paper suggests a definition of "know who" as a modality using Grove-Halpern semantics of names. It also introduces a logical system that describes the interplay between modalities "knows who", "knows", and "for all agents". The main technical result is a completeness theorem for the proposed system.
Computing Machinery and Knowledge
Authors: Raymond Anneborg
Date: 2020-10-31 00:00:00
Tasks: AI Agent

The purpose of this paper is to discuss the possibilities for computing machinery, or AI agents, to know and to possess knowledge. This is done mainly from a virtue epistemology perspective and definition of knowledge. However, this inquiry also shed light on the human condition, what it means for a human to know, and to possess knowledge. The paper argues that it is possible for an AI agent to know and examines this from both current state-of-the-art in artificial intelligence as well as from the perspective of what the future AI development might bring in terms of superintelligent AI agents.
A Unified Model for the Two-stage Offline-then-Online Resource Allocation
Authors: Yifan Xu, Pan Xu, Jianping Pan, Jun Tao
Date: 2020-12-12 00:00:00
Tasks: Decision Making

With the popularity of the Internet, traditional offline resource allocation has evolved into a new form, called online resource allocation. It features the online arrivals of agents in the system and the real-time decision-making requirement upon the arrival of each online agent. Both offline and online resource allocation have wide applications in various real-world matching markets ranging from ridesharing to crowdsourcing. There are some emerging applications such as rebalancing in bike sharing and trip-vehicle dispatching in ridesharing, which involve a two-stage resource allocation process. The process consists of an offline phase and another sequential online phase, and both phases compete for the same set of resources. In this paper, we propose a unified model which incorporates both offline and online resource allocation into a single framework. Our model assumes non-uniform and known arrival distributions for online agents in the second online phase, which can be learned from historical data. We propose a parameterized linear programming (LP)-based algorithm, which is shown to be at most a constant factor of $1/4$ from the optimal. Experimental results on the real dataset show that our LP-based approaches outperform the LP-agnostic heuristics in terms of robustness and effectiveness.
Trading the System Efficiency for the Income Equality of Drivers in Rideshare
Authors: Yifan Xu, Pan Xu
Date: 2020-12-12 00:00:00
Tasks: Fairness, valid

Several scientific studies have reported the existence of the income gap among rideshare drivers based on demographic factors such as gender, age, race, etc. In this paper, we study the income inequality among rideshare drivers due to discriminative cancellations from riders, and the tradeoff between the income inequality (called fairness objective) with the system efficiency (called profit objective). We proposed an online bipartite-matching model where riders are assumed to arrive sequentially following a distribution known in advance. The highlight of our model is the concept of acceptance rate between any pair of driver-rider types, where types are defined based on demographic factors. Specially, we assume each rider can accept or cancel the driver assigned to her, each occurs with a certain probability which reflects the acceptance degree from the rider type towards the driver type. We construct a bi-objective linear program as a valid benchmark and propose two LP-based parameterized online algorithms. Rigorous online competitive ratio analysis is offered to demonstrate the flexibility and efficiency of our online algorithms in balancing the two conflicting goals, promotions of fairness and profit. Experimental results on a real-world dataset are provided as well, which confirm our theoretical predictions.