pwc-archive/papers-with-abstracts huggingface dataset:
A Tutorial on Ultra-Reliable and Low-Latency Communications in 6G: Integrating Domain Knowledge into Deep Learning
Authors: Changyang She, Chengjian Sun, Zhouyou Gu, Yonghui Li, Chenyang Yang, H. Vincent Poor, Branka Vucetic
Date: 2020-09-13 00:00:00
Tasks: Decision Making, Decision Making Under Uncertainty, Deep Learning, Deep Reinforcement Learning

As one of the key communication scenarios in the 5th and also the 6th generation (6G) of mobile communication networks, ultra-reliable and low-latency communications (URLLC) will be central for the development of various emerging mission-critical applications. State-of-the-art mobile communication systems do not fulfill the end-to-end delay and overall reliability requirements of URLLC. In particular, a holistic framework that takes into account latency, reliability, availability, scalability, and decision making under uncertainty is lacking. Driven by recent breakthroughs in deep neural networks, deep learning algorithms have been considered as promising ways of developing enabling technologies for URLLC in future 6G networks. This tutorial illustrates how domain knowledge (models, analytical tools, and optimization frameworks) of communications and networking can be integrated into different kinds of deep learning algorithms for URLLC. We first provide some background of URLLC and review promising network architectures and deep learning frameworks for 6G. To better illustrate how to improve learning algorithms with domain knowledge, we revisit model-based analytical tools and cross-layer optimization frameworks for URLLC. Following that, we examine the potential of applying supervised/unsupervised deep learning and deep reinforcement learning in URLLC and summarize related open problems. Finally, we provide simulation and experimental results to validate the effectiveness of different learning algorithms and discuss future directions.
The Platform Design Problem
Authors: Christos Papadimitriou, Kiran Vodrahalli, Mihalis Yannakakis
Date: 2020-09-13 00:00:00

On-line firms deploy suites of software platforms, where each platform is designed to interact with users during a certain activity, such as browsing, chatting, socializing, emailing, driving, etc. The economic and incentive structure of this exchange, as well as its algorithmic nature, have not been explored to our knowledge. We model this interaction as a Stackelberg game between a Designer and one or more Agents. We model an Agent as a Markov chain whose states are activities; we assume that the Agent's utility is a linear function of the steady-state distribution of this chain. The Designer may design a platform for each of these activities/states; if a platform is adopted by the Agent, the transition probabilities of the Markov chain are affected, and so is the objective of the Agent. The Designer's utility is a linear function of the steady state probabilities of the accessible states minus the development cost of the platforms. The underlying optimization problem of the Agent -- how to choose the states for which to adopt the platform -- is an MDP. If this MDP has a simple yet plausible structure (the transition probabilities from one state to another only depend on the target state and the recurrent probability of the current state) the Agent's problem can be solved by a greedy algorithm. The Designer's optimization problem (designing a custom suite for the Agent so as to optimize, through the Agent's optimum reaction, the Designer's revenue), is NP-hard to approximate within any finite ratio; however, the special case, while still NP-hard, has an FPTAS. These results generalize from a single Agent to a distribution of Agents with finite support, as well as to the setting where the Designer must find the best response to the existing strategies of other Designers. We discuss other implications of our results and directions of future research.
Super Resolution of Arterial Spin Labeling MR Imaging Using Unsupervised Multi-Scale Generative Adversarial Network
Authors: Jianan Cui, Kuang Gong, Paul Han, Huafeng Liu, Quanzheng Li
Date: 2020-09-14 00:00:00
Tasks: Generative Adversarial Network, SSIM, Super-Resolution

Arterial spin labeling (ASL) magnetic resonance imaging (MRI) is a powerful imaging technology that can measure cerebral blood flow (CBF) quantitatively. However, since only a small portion of blood is labeled compared to the whole tissue volume, conventional ASL suffers from low signal-to-noise ratio (SNR), poor spatial resolution, and long acquisition time. In this paper, we proposed a super-resolution method based on a multi-scale generative adversarial network (GAN) through unsupervised training. The network only needs the low-resolution (LR) ASL image itself for training and the T1-weighted image as the anatomical prior. No training pairs or pre-training are needed. A low-pass filter guided item was added as an additional loss to suppress the noise interference from the LR ASL image. After the network was trained, the super-resolution (SR) image was generated by supplying the upsampled LR ASL image and corresponding T1-weighted image to the generator of the last layer. Performance of the proposed method was evaluated by comparing the peak signal-to-noise ratio (PSNR) and structural similarity index (SSIM) using normal-resolution (NR) ASL image (5.5 min acquisition) and high-resolution (HR) ASL image (44 min acquisition) as the ground truth. Compared to the nearest, linear, and spline interpolation methods, the proposed method recovers more detailed structure information, reduces the image noise visually, and achieves the highest PSNR and SSIM when using HR ASL image as the ground-truth.
Multi-Agent Reinforcement Learning in Cournot Games
Authors: Yuanyuan Shi, Baosen Zhang
Date: 2020-09-14 00:00:00
Tasks: continuous-control, Continuous Control, Multi-agent Reinforcement Learning, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

In this work, we study the interaction of strategic agents in continuous action Cournot games with limited information feedback. Cournot game is the essential market model for many socio-economic systems where agents learn and compete without the full knowledge of the system or each other. We consider the dynamics of the policy gradient algorithm, which is a widely adopted continuous control reinforcement learning algorithm, in concave Cournot games. We prove the convergence of policy gradient dynamics to the Nash equilibrium when the price function is linear or the number of agents is two. This is the first result (to the best of our knowledge) on the convergence property of learning algorithms with continuous action spaces that do not fall in the no-regret class.
Comprehensive Comparison of Deep Learning Models for Lung and COVID-19 Lesion Segmentation in CT scans
Authors: Paschalis Bizopoulos, Nicholas Vretos, Petros Daras
Date: 2020-09-10 00:00:00
Tasks: GPU, Image Segmentation, Lesion Segmentation, Medical Image Segmentation, Segmentation, Semantic Segmentation

Recently there has been an explosion in the use of Deep Learning (DL) methods for medical image segmentation. However the field's reliability is hindered by the lack of a common base of reference for accuracy/performance evaluation and the fact that previous research uses different datasets for evaluation. In this paper, an extensive comparison of DL models for lung and COVID-19 lesion segmentation in Computerized Tomography (CT) scans is presented, which can also be used as a benchmark for testing medical image segmentation models. Four DL architectures (Unet, Linknet, FPN, PSPNet) are combined with 25 randomly initialized and pretrained encoders (variations of VGG, DenseNet, ResNet, ResNext, DPN, MobileNet, Xception, Inception-v4, EfficientNet), to construct 200 tested models. Three experimental setups are conducted for lung segmentation, lesion segmentation and lesion segmentation using the original lung masks. A public COVID-19 dataset with 100 CT scan images (80 for train, 20 for validation) is used for training/validation and a different public dataset consisting of 829 images from 9 CT scan volumes for testing. Multiple findings are provided including the best architecture-encoder models for each experiment as well as mean Dice results for each experiment, architecture and encoder independently. Finally, the upper bounds improvements when using lung masks as a preprocessing step or when using pretrained models are quantified. The source code and 600 pretrained models for the three experiments are provided, suitable for fine-tuning in experimental setups without GPU capabilities.
Supervised learning for the prediction of firm dynamics
Authors: Falco J. Bargagli-Stoffi, Jan Niederreiter, Massimo Riccaboni
Date: 2020-09-11 00:00:00
Tasks: Prediction

Thanks to the increasing availability of granular, yet high-dimensional, firm level data, machine learning (ML) algorithms have been successfully applied to address multiple research questions related to firm dynamics. Especially supervised learning (SL), the branch of ML dealing with the prediction of labelled outcomes, has been used to better predict firms' performance. In this contribution, we will illustrate a series of SL approaches to be used for prediction tasks, relevant at different stages of the company life cycle. The stages we will focus on are (i) startup and innovation, (ii) growth and performance of companies, and (iii) firms exit from the market. First, we review SL implementations to predict successful startups and R&D projects. Next, we describe how SL tools can be used to analyze company growth and performance. Finally, we review SL applications to better forecast financial distress and company failure. In the concluding Section, we extend the discussion of SL methods in the light of targeted policies, result interpretability, and causality.
Deconstructing Legal Text_Object Oriented Design in Legal Adjudication
Authors: Megan Ma, Dmitriy Podkopaev, Avalon Campbell-Cousins, Adam Nicholas
Date: 2020-09-13 00:00:00
Tasks: Descriptive, Legal Reasoning, Translation

Rules are pervasive in the law. In the context of computer engineering, the translation of legal text to algorithmic form is seemingly direct. In large part, law may be a ripe field for expert systems and machine learning. For engineers, existing law appears formulaic and logically reducible to "if, then" statements. The underlying assumption is that the legal language is both self-referential and universal. Moreover, description is considered distinct from interpretation; that in describing the law, the language is seen as quantitative and objectifiable. Nevertheless, is descriptive formal language purely dissociative? From the logic machine of the 1970s to the modern fervor for artificial intelligence (AI), governance by numbers is making a persuasive return. Could translation be possible? The project follows a fundamentally semantic conundrum: what is the significance of "meaning" in legal language? The project, therefore, tests translation by deconstructing sentences from existing legal judgments to their constituent factors. Definitions are then extracted in accordance with the interpretations of the judges. The intent is to build an expert system predicated on alleged rules of legal reasoning. The authors apply both linguistic modelling and natural language processing technology to parse the legal judgments. The project extends beyond prior research in the area, combining a broadly statistical model of context with the relative precision of syntactic structure. The preliminary hypothesis is that, by analyzing the components of legal language with a variety of techniques, we can begin to translate law to numerical form.
Double-Wing Mixture of Experts for Streaming Recommendations
Authors: Yan Zhao, Shoujin Wang, Yan Wang, Hongwei Liu, Weizhe Zhang
Date: 2020-09-14 00:00:00
Tasks: Ensemble Learning, Mixture-of-Experts, Recommendation Systems

Streaming Recommender Systems (SRSs) commonly train recommendation models on newly received data only to address user preference drift, i.e., the changing user preferences towards items. However, this practice overlooks the long-term user preferences embedded in historical data. More importantly, the common heterogeneity in data stream greatly reduces the accuracy of streaming recommendations. The reason is that different preferences (or characteristics) of different types of users (or items) cannot be well learned by a unified model. To address these two issues, we propose a Variational and Reservoir-enhanced Sampling based Double-Wing Mixture of Experts framework, called VRS-DWMoE, to improve the accuracy of streaming recommendations. In VRS-DWMoE, we first devise variational and reservoir-enhanced sampling to wisely complement new data with historical data, and thus address the user preference drift issue while capturing long-term user preferences. After that, we propose a Double-Wing Mixture of Experts (DWMoE) model to first effectively learn heterogeneous user preferences and item characteristics, and then make recommendations based on them. Specifically, DWMoE contains two Mixture of Experts (MoE, an effective ensemble learning model) to learn user preferences and item characteristics, respectively. Moreover, the multiple experts in each MoE learn the preferences (or characteristics) of different types of users (or items) where each expert specializes in one underlying type. Extensive experiments demonstrate that VRS-DWMoE consistently outperforms the state-of-the-art SRSs.
Characterizing Twitter Interaction during COVID-19 pandemic using Complex Networks and Text Mining
Authors: Josimar E. Chire-Saire
Date: 2020-09-11 00:00:00

The outbreak of covid-19 started many months ago, the reported origin was in Wuhan Market, China. Fastly, this virus was propagated to other countries because the access to international travels is affordable and many countries have a distance of some flight hours, besides borders were a constant flow of people. By the other hand, Internet users have the habits of sharing content using Social Networks and issues, problems, thoughts about Covdid-19 were not an exception. Therefore, it is possible to analyze Social Network interaction from one city, country to understand the impact generated by this global issue. South America is one region with developing countries with challenges to face related to Politics, Economy, Public Health and other. Therefore, the scope of this paper is to analyze the interaction on Twitter of South American countries and characterize the flow of data through the users using Complex Network representation and Text Mining. The preliminary experiments introduces the idea of existence of patterns, similar to Complex Systems. Besides, the degree distribution confirm the idea of having a System and visualization of Adjacency Matrices show the presence of users' group publishing and interacting together during the time, there is a possibility of identification of robots sending posts constantly.
An Ensemble of Simple Convolutional Neural Network Models for MNIST Digit Recognition
Authors: Sanghyeon An, Minjun Lee, Sanglee Park, Heerin Yang, Jungmin So
Date: 2020-08-12 00:00:00
Tasks: image-classification, Image Classification, Translation

We report that a very high accuracy on the MNIST test set can be achieved by using simple convolutional neural network (CNN) models. We use three different models with 3x3, 5x5, and 7x7 kernel size in the convolution layers. Each model consists of a set of convolution layers followed by a single fully connected layer. Every convolution layer uses batch normalization and ReLU activation, and pooling is not used. Rotation and translation is used to augment training data, which is frequently used in most image classification tasks. A majority voting using the three models independently trained on the training data set can achieve up to 99.87% accuracy on the test set, which is one of the state-of-the-art results. A two-layer ensemble, a heterogeneous ensemble of three homogeneous ensemble networks, can achieve up to 99.91% test accuracy. The results can be reproduced by using the code at: https://github.com/ansh941/MnistSimpleCNN
Discriminative Learning of Sum-Product Networks
Authors: Robert Gens
Date: 2011-01-01 00:00:00
Tasks: image-classification, Image Classification

Sum-product networks are a new deep architecture that can perform fast, exact inference on high-treewidth models. Only generative methods for training SPNs
have been proposed to date. In this paper, we present the first discriminative
training algorithms for SPNs, combining the high accuracy of the former with
the representational power and tractability of the latter. We show that the class
of tractable discriminative SPNs is broader than the class of tractable generative
ones, and propose an efficient backpropagation-style algorithm for computing the
gradient of the conditional log likelihood. Standard gradient descent suffers from
the diffusion problem, but networks with many layers can be learned reliably using “hard” gradient descent, where marginal inference is replaced by MPE inference (i.e., inferring the most probable state of the non-evidence variables). The
resulting updates have a simple and intuitive form. We test discriminative SPNs
on standard image classification tasks. We obtain the best results to date on the
CIFAR-10 dataset, using fewer features than prior methods with an SPN architecture that learns local image structure discriminatively. We also report the highest
published test accuracy on STL-10 even though we only use the labeled portion
of the dataset.
Understanding Deformable Alignment in Video Super-Resolution
Authors: Kelvin C. K. Chan, Xintao Wang, Ke Yu, Chao Dong, Chen Change Loy
Date: 2020-09-15 00:00:00
Tasks: Diversity, Optical Flow Estimation, Super-Resolution, Video Super-Resolution

Deformable convolution, originally proposed for the adaptation to geometric variations of objects, has recently shown compelling performance in aligning multiple frames and is increasingly adopted for video super-resolution. Despite its remarkable performance, its underlying mechanism for alignment remains unclear. In this study, we carefully investigate the relation between deformable alignment and the classic flow-based alignment. We show that deformable convolution can be decomposed into a combination of spatial warping and convolution. This decomposition reveals the commonality of deformable alignment and flow-based alignment in formulation, but with a key difference in their offset diversity. We further demonstrate through experiments that the increased diversity in deformable alignment yields better-aligned features, and hence significantly improves the quality of video super-resolution output. Based on our observations, we propose an offset-fidelity loss that guides the offset learning with optical flow. Experiments show that our loss successfully avoids the overflow of offsets and alleviates the instability problem of deformable alignment. Aside from the contributions to deformable alignment, our formulation inspires a more flexible approach to introduce offset diversity to flow-based alignment, improving its performance.
Don’t Wait, Just Weight: Improving Unsupervised Representations by Learning Goal-Driven Instance Weights
Authors: Linus Ericsson
Date: 2020-06-22 00:00:00
Tasks: Image Classification, Meta-Learning, Self-Supervised Learning

In the absence of large labelled datasets, self-supervised learning techniques
can boost performance by learning useful representations from unlabelled data,
which is often more readily available. However, there is often a domain shift
between the unlabelled collection and the downstream target problem data. We
show that by learning Bayesian instance weights for the unlabelled data, we
can improve the downstream classification accuracy by prioritising the most
useful instances. Additionally, we show that the training time can be reduced by
discarding unnecessary datapoints. Our method, BetaDataWeighter is evaluated
using the popular self-supervised rotation prediction task on STL-10 and Visual
Decathlon. We compare to related instance weighting schemes, both hand-designed
heuristics and meta-learning, as well as conventional self-supervised learning.
BetaDataWeighter achieves both the highest average accuracy and rank across
datasets, and on STL-10 it prunes up to 78% of unlabelled images without significant
loss in accuracy, corresponding to over 50% reduction in training time.
Probabilistic Structural Latent Representation for Unsupervised Embedding
Authors: Mang Ye, Jianbing Shen∗
Date: 2020-06-22 00:00:00
Tasks: Data Augmentation

Unsupervised embedding learning aims at extracting
low-dimensional visually meaningful representations from
large-scale unlabeled images, which can then be directly
used for similarity-based search. This task faces two major
challenges: 1) mining positive supervision from highly similar fine-grained classes and 2) generating to unseen testing categories. To tackle these issues, this paper proposes a
probabilistic structural latent representation (PSLR), which
incorporates an adaptable softmax embedding to approximate the positive concentrated and negative instance separated properties in the graph latent space. It improves
the discriminability by enlarging the positive/negative difference without introducing any additional computational
cost while maintaining high learning efficiency. To address
the limited supervision using data augmentation, a smooth
variational reconstruction loss is introduced by modeling
the intra-instance variance, which improves the robustness.
Extensive experiments demonstrate the superiority of PSLR
over state-of-the-art unsupervised methods on both seen
and unseen categories with cosine similarity. Code is available at https://github.com/mangye16/PSLR
Multi-Referenced Training for Dialogue Response Generation
Authors: Tianyu Zhao, Tatsuya Kawahara
Date: 2020-09-15 00:00:00
Proceeding: SIGDIAL (ACL) 2021 7
Tasks: Response Generation

In open-domain dialogue response generation, a dialogue context can be continued with diverse responses, and the dialogue models should capture such one-to-many relations. In this work, we first analyze the training objective of dialogue models from the view of Kullback-Leibler divergence (KLD) and show that the gap between the real world probability distribution and the single-referenced data's probability distribution prevents the model from learning the one-to-many relations efficiently. Then we explore approaches to multi-referenced training in two aspects. Data-wise, we generate diverse pseudo references from a powerful pretrained model to build multi-referenced data that provides a better approximation of the real-world distribution. Model-wise, we propose to equip variational models with an expressive prior, named linear Gaussian model (LGM). Experimental results of automated evaluation and human evaluation show that the methods yield significant improvements over baselines. We will release our code and data in https://github.com/ZHAOTING/dialog-processing.