pwc-archive/papers-with-abstracts huggingface dataset:
Value of structural health information in partially observable stochastic environments
Authors: C. P. Andriotis, K. G. Papakonstantinou, E. N. Chatzi
Date: 2019-12-28 00:00:00
Tasks: Decision Making, Decision Making Under Uncertainty, Scheduling, Structural Health Monitoring

Efficient integration of uncertain observations with decision-making optimization is key for prescribing informed intervention actions, able to preserve structural safety of deteriorating engineering systems. To this end, it is necessary that scheduling of inspection and monitoring strategies be objectively performed on the basis of their expected value-based gains that, among others, reflect quantitative metrics such as the Value of Information (VoI) and the Value of Structural Health Monitoring (VoSHM). In this work, we introduce and study the theoretical and computational foundations of the above metrics within the context of Partially Observable Markov Decision Processes (POMDPs), thus alluding to a broad class of decision-making problems of partially observable stochastic deteriorating environments that can be modeled as POMDPs. Step-wise and life-cycle VoI and VoSHM definitions are devised and their bounds are analyzed as per the properties stemming from the Bellman equation and the resulting optimal value function. It is shown that a POMDP policy inherently leverages the notion of VoI to guide observational actions in an optimal way at every decision step, and that the permanent or intermittent information provided by SHM or inspection visits, respectively, can only improve the cost of this policy in the long-term, something that is not necessarily true under locally optimal policies, typically adopted in decision-making of structures and infrastructure. POMDP solutions are derived based on point-based value iteration methods, and the various definitions are quantified in stationary and non-stationary deteriorating environments, with both infinite and finite planning horizons, featuring single- or multi-component engineering systems.
Learning to segment images with classification labels
Authors: Ozan Ciga, Anne L. Martel
Date: 2019-12-28 00:00:00
Tasks: Classification, General Classification, Segmentation, whole slide images

Two of the most common tasks in medical imaging are classification and segmentation. Either task requires labeled data annotated by experts, which is scarce and expensive to collect. Annotating data for segmentation is generally considered to be more laborious as the annotator has to draw around the boundaries of regions of interest, as opposed to assigning image patches a class label. Furthermore, in tasks such as breast cancer histopathology, any realistic clinical application often includes working with whole slide images, whereas most publicly available training data are in the form of image patches, which are given a class label. We propose an architecture that can alleviate the requirements for segmentation-level ground truth by making use of image-level labels to reduce the amount of time spent on data curation. In addition, this architecture can help unlock the potential of previously acquired image-level datasets on segmentation tasks by annotating a small number of regions of interest. In our experiments, we show using only one segmentation-level annotation per class, we can achieve performance comparable to a fully annotated dataset.
Applications of Unsupervised Deep Transfer Learning to Intelligent Fault Diagnosis: A Survey and Comparative Study
Authors: Zhibin Zhao, Qiyang Zhang, Xiaolei Yu, Chuang Sun, Shibin Wang, Ruqiang Yan, Xuefeng Chen
Date: 2019-12-28 00:00:00
Tasks: Fault Diagnosis, Representation Learning, Transfer Learning

Recent progress on intelligent fault diagnosis (IFD) has greatly depended on deep representation learning and plenty of labeled data. However, machines often operate with various working conditions or the target task has different distributions with the collected data used for training (the domain shift problem). Besides, the newly collected test data in the target domain are usually unlabeled, leading to unsupervised deep transfer learning based (UDTL-based) IFD problem. Although it has achieved huge development, a standard and open source code framework as well as a comparative study for UDTL-based IFD are not yet established. In this paper, we construct a new taxonomy and perform a comprehensive review of UDTL-based IFD according to different tasks. Comparative analysis of some typical methods and datasets reveals some open and essential issues in UDTL-based IFD which are rarely studied, including transferability of features, influence of backbones, negative transfer, physical priors, etc. To emphasize the importance and reproducibility of UDTL-based IFD, the whole test framework will be released to the research community to facilitate future research. In summary, the released framework and comparative study can serve as an extended interface and basic results to carry out new studies on UDTL-based IFD. The code framework is available at \url{https://github.com/ZhaoZhibin/UDTL}.
NAS evaluation is frustratingly hard
Authors: Antoine Yang, Pedro M. Esperança, Fabio M. Carlucci
Date: 2019-12-28 00:00:00
Proceeding: ICLR 2020 1
Tasks: Image Classification, Neural Architecture Search

Neural Architecture Search (NAS) is an exciting new field which promises to be as much as a game-changer as Convolutional Neural Networks were in 2012. Despite many great works leading to substantial improvements on a variety of tasks, comparison between different methods is still very much an open issue. While most algorithms are tested on the same datasets, there is no shared experimental protocol followed by all. As such, and due to the under-use of ablation studies, there is a lack of clarity regarding why certain methods are more effective than others. Our first contribution is a benchmark of $8$ NAS methods on $5$ datasets. To overcome the hurdle of comparing methods with different search spaces, we propose using a method's relative improvement over the randomly sampled average architecture, which effectively removes advantages arising from expertly engineered search spaces or training protocols. Surprisingly, we find that many NAS techniques struggle to significantly beat the average architecture baseline. We perform further experiments with the commonly used DARTS search space in order to understand the contribution of each component in the NAS pipeline. These experiments highlight that: (i) the use of tricks in the evaluation protocol has a predominant impact on the reported performance of architectures; (ii) the cell-based search space has a very narrow accuracy range, such that the seed has a considerable impact on architecture rankings; (iii) the hand-designed macro-structure (cells) is more important than the searched micro-structure (operations); and (iv) the depth-gap is a real phenomenon, evidenced by the change in rankings between $8$ and $20$ cell architectures. To conclude, we suggest best practices, that we hope will prove useful for the community and help mitigate current NAS pitfalls. The code used is available at https://github.com/antoyang/NAS-Benchmark.
Weak Supervision for Fake News Detection via Reinforcement Learning
Authors: Yaqing Wang, Weifeng Yang, Fenglong Ma, Jin Xu, Bin Zhong, Qiang Deng, Jing Gao
Date: 2019-12-28 00:00:00
Tasks: Articles, Fake News Detection, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

Today social media has become the primary source for news. Via social media platforms, fake news travel at unprecedented speeds, reach global audiences and put users and communities at great risk. Therefore, it is extremely important to detect fake news as early as possible. Recently, deep learning based approaches have shown improved performance in fake news detection. However, the training of such models requires a large amount of labeled data, but manual annotation is time-consuming and expensive. Moreover, due to the dynamic nature of news, annotated samples may become outdated quickly and cannot represent the news articles on newly emerged events. Therefore, how to obtain fresh and high-quality labeled samples is the major challenge in employing deep learning models for fake news detection. In order to tackle this challenge, we propose a reinforced weakly-supervised fake news detection framework, i.e., WeFEND, which can leverage users' reports as weak supervision to enlarge the amount of training data for fake news detection. The proposed framework consists of three main components: the annotator, the reinforced selector and the fake news detector. The annotator can automatically assign weak labels for unlabeled news based on users' reports. The reinforced selector using reinforcement learning techniques chooses high-quality samples from the weakly labeled data and filters out those low-quality ones that may degrade the detector's prediction performance. The fake news detector aims to identify fake news based on the news content. We tested the proposed framework on a large collection of news articles published via WeChat official accounts and associated user reports. Extensive experiments on this dataset show that the proposed WeFEND model achieves the best performance compared with the state-of-the-art methods.
Tha3aroon at NSURL-2019 Task 8: Semantic Question Similarity in Arabic
Authors: Ali Fadel, Ibraheem Tuffaha, Mahmoud Al-Ayyoub
Date: 2019-12-28 00:00:00
Proceeding: NSURL 2019 9
Tasks: Data Augmentation, Question Similarity

In this paper, we describe our team's effort on the semantic text question similarity task of NSURL 2019. Our top performing system utilizes several innovative data augmentation techniques to enlarge the training data. Then, it takes ELMo pre-trained contextual embeddings of the data and feeds them into an ON-LSTM network with self-attention. This results in sequence representation vectors that are used to predict the relation between the question pairs. The model is ranked in the 1st place with 96.499 F1-score (same as the second place F1-score) and the 2nd place with 94.848 F1-score (differs by 1.076 F1-score from the first place) on the public and private leaderboards, respectively.
Detecting Out-of-Distribution Examples with In-distribution Examples and Gram Matrices
Authors: Chandramouli Shama Sastry, Sageev Oore
Date: 2019-12-28 00:00:00
Tasks: Out-of-Distribution Detection, Out of Distribution (OOD) Detection

When presented with Out-of-Distribution (OOD) examples, deep neural networks yield confident, incorrect predictions. Detecting OOD examples is challenging, and the potential risks are high. In this paper, we propose to detect OOD examples by identifying inconsistencies between activity patterns and class predicted. We find that characterizing activity patterns by Gram matrices and identifying anomalies in gram matrix values can yield high OOD detection rates. We identify anomalies in the gram matrices by simply comparing each value with its respective range observed over the training data. Unlike many approaches, this can be used with any pre-trained softmax classifier and does not require access to OOD data for fine-tuning hyperparameters, nor does it require OOD access for inferring parameters. The method is applicable across a variety of architectures and vision datasets and, for the important and surprisingly hard task of detecting far-from-distribution out-of-distribution examples, it generally performs better than or equal to state-of-the-art OOD detection methods (including those that do assume access to OOD examples).
Implicit supervision for fault detection and segmentation of emerging fault types with Deep Variational Autoencoders
Authors: Manuel Arias Chao, Bryan T. Adey, Olga Fink
Date: 2019-12-28 00:00:00
Tasks: Clustering, Fault Detection, One-Class Classification, One-class classifier, Open Set Learning, Segmentation

Data-driven fault diagnostics of safety-critical systems often faces the challenge of a complete lack of labeled data associated with faulty system conditions (i.e., fault types) at training time. Since an unknown number and nature of fault types can arise during deployment, data-driven fault diagnostics in this scenario is an open-set learning problem. Most of the algorithms for open-set diagnostics are one-class classification and unsupervised algorithms that do not leverage all the available labeled and unlabeled data in the learning algorithm. As a result, their fault detection and segmentation performance (i.e., identifying and separating faults of different types) are sub-optimal. With this work, we propose training a variational autoencoder (VAE) with labeled and unlabeled samples while inducing implicit supervision on the latent representation of the healthy conditions. This, together with a modified sampling process of VAE, creates a compact and informative latent representation that allows good detection and segmentation of unseen fault types using existing one-class and clustering algorithms. We refer to the proposed methodology as "knowledge induced variational autoencoder with adaptive sampling" (KIL-AdaVAE). The fault detection and segmentation capabilities of the proposed methodology are demonstrated in a new simulated case study using the Advanced Geared Turbofan 30000 (AGTF30) dynamical model under real flight conditions. In an extensive comparison, we demonstrate that the proposed method outperforms other learning strategies (supervised learning, supervised learning with embedding and semi-supervised learning) and deep learning algorithms, yielding significant performance improvements on fault detection and fault segmentation.
Alleviation of Gradient Exploding in GANs: Fake Can Be Real
Authors: Song Tao, Jia Wang
Date: 2019-12-28 00:00:00
Proceeding: CVPR 2020 6

In order to alleviate the notorious mode collapse phenomenon in generative adversarial networks (GANs), we propose a novel training method of GANs in which certain fake samples are considered as real ones during the training process. This strategy can reduce the gradient value that generator receives in the region where gradient exploding happens. We show the process of an unbalanced generation and a vicious circle issue resulted from gradient exploding in practical training, which explains the instability of GANs. We also theoretically prove that gradient exploding can be alleviated by penalizing the difference between discriminator outputs and fake-as-real consideration for very close real and fake samples. Accordingly, Fake-As-Real GAN (FARGAN) is proposed with a more stable training process and a more faithful generated distribution. Experiments on different datasets verify our theoretical analysis.
SLM Lab: A Comprehensive Benchmark and Modular Software Framework for Reproducible Deep Reinforcement Learning
Authors: Keng Wah Loon, Laura Graesser, Milan Cvitkovic
Date: 2019-12-28 00:00:00
Tasks: Atari Games, Deep Reinforcement Learning, OpenAI Gym, Reinforcement Learning, Reinforcement Learning (RL)

We introduce SLM Lab, a software framework for reproducible reinforcement learning (RL) research. SLM Lab implements a number of popular RL algorithms, provides synchronous and asynchronous parallel experiment execution, hyperparameter search, and result analysis. RL algorithms in SLM Lab are implemented in a modular way such that differences in algorithm performance can be confidently ascribed to differences between algorithms, not between implementations. In this work we present the design choices behind SLM Lab and use it to produce a comprehensive single-codebase RL algorithm benchmark. In addition, as a consequence of SLM Lab's modular design, we introduce and evaluate a discrete-action variant of the Soft Actor-Critic algorithm (Haarnoja et al., 2018) and a hybrid synchronous/asynchronous training method for RL agents.
Robust Cross-lingual Embeddings from Parallel Sentences
Authors: Ali Sabet, Prakhar Gupta, Jean-Baptiste Cordonnier, Robert West, Martin Jaggi
Date: 2019-12-28 00:00:00
Tasks: Cross-Lingual Document Classification, Cross-Lingual Word Embeddings, Document Classification, Retrieval, Sentence, Sentence Retrieval, Translation, Word Embeddings, Word Translation, Zero-shot Cross-Lingual Document Classification

Recent advances in cross-lingual word embeddings have primarily relied on mapping-based methods, which project pretrained word embeddings from different languages into a shared space through a linear transformation. However, these approaches assume word embedding spaces are isomorphic between different languages, which has been shown not to hold in practice (S{\o}gaard et al., 2018), and fundamentally limits their performance. This motivates investigating joint learning methods which can overcome this impediment, by simultaneously learning embeddings across languages via a cross-lingual term in the training objective. We propose a bilingual extension of the CBOW method which leverages sentence-aligned corpora to obtain robust cross-lingual word and sentence representations. Our approach significantly improves cross-lingual sentence retrieval performance over all other approaches while maintaining parity with the current state-of-the-art methods on word-translation. It also achieves parity with a deep RNN method on a zero-shot cross-lingual document classification task, requiring far fewer computational resources for training and inference. As an additional advantage, our bilingual method leads to a much more pronounced improvement in the the quality of monolingual word vectors compared to other competing methods.
Short-Term Load Forecasting Using AMI Data
Authors: Haris Mansoor, Sarwan Ali, Imdadullah Khan, Naveed Arshad, Muhammad Asad Khan, Safiullah Faizullah
Date: 2019-12-28 00:00:00
Tasks: Load Forecasting

Accurate short-term load forecasting is essential for the efficient operation of the power sector. Forecasting load at a fine granularity such as hourly loads of individual households is challenging due to higher volatility and inherent stochasticity. At the aggregate levels, such as monthly load at a grid, the uncertainties and fluctuations are averaged out; hence predicting load is more straightforward. This paper proposes a method called Forecasting using Matrix Factorization (\textsc{fmf}) for short-term load forecasting (\textsc{stlf}). \textsc{fmf} only utilizes historical data from consumers' smart meters to forecast future loads (does not use any non-calendar attributes, consumers' demographics or activity patterns information, etc.) and can be applied to any locality. A prominent feature of \textsc{fmf} is that it works at any level of user-specified granularity, both in the temporal (from a single hour to days) and spatial dimensions (a single household to groups of consumers). We empirically evaluate \textsc{fmf} on three benchmark datasets and demonstrate that it significantly outperforms the state-of-the-art methods in terms of load forecasting. The computational complexity of \textsc{fmf} is also substantially less than known methods for \textsc{stlf} such as long short-term memory neural networks, random forest, support vector machines, and regression trees.
Energy-based Graph Convolutional Networks for Scoring Protein Docking Models
Authors: Yue Cao, Yang shen
Date: 2019-12-28 00:00:00

Structural information about protein-protein interactions, often missing at the interactome scale, is important for mechanistic understanding of cells and rational discovery of therapeutics. Protein docking provides a computational alternative to predict such information. However, ranking near-native docked models high among a large number of candidates, often known as the scoring problem, remains a critical challenge. Moreover, estimating model quality, also known as the quality assessment problem, is rarely addressed in protein docking. In this study the two challenging problems in protein docking are regarded as relative and absolute scoring, respectively, and addressed in one physics-inspired deep learning framework. We represent proteins' and encounter complexes' 3D structures as intra- and inter-molecular residue contact graphs with atom-resolution node and edge features. And we propose a novel graph convolutional kernel that pool interacting nodes' features through edge features so that generalized interaction energies can be learned directly from graph data. The resulting energy-based graph convolutional networks (EGCN) with multi-head attention are trained to predict intra- and inter-molecular energies, binding affinities, and quality measures (interface RMSD) for encounter complexes. Compared to a state-of-the-art scoring function for model ranking, EGCN has significantly improved ranking for a CAPRI test set involving homology docking; and is comparable for Score_set, a CAPRI benchmark set generated by diverse community-wide docking protocols not known to training data. For Score_set quality assessment, EGCN shows about 27% improvement to our previous efforts. Directly learning from 3D structure data in graph representation, EGCN represents the first successful development of graph convolutional networks for protein docking.
Arachne: Search Based Repair of Deep Neural Networks
Authors: Jeongju Sohn, Sungmin Kang, Shin Yoo
Date: 2019-12-28 00:00:00
Tasks: Fairness, Gender Classification, Program Repair

The rapid and widespread adoption of Deep Neural Networks (DNNs) has called for ways to test their behaviour, and many testing approaches have successfully revealed misbehaviour of DNNs. However, it is relatively unclear what one can do to correct such behaviour after revelation, as retraining involves costly data collection and does not guarantee to fix the underlying issue. This paper introduces Arachne, a novel program repair technique for DNNs, which directly repairs DNNs using their input-output pairs as a specification. Arachne localises neural weights on which it can generate effective patches and uses Differential Evolution to optimise the localised weights and correct the misbehaviour. An empirical study using different benchmarks shows that Arachne can fix specific misclassifications of a DNN without reducing general accuracy significantly. On average, patches generated by Arachne generalise to 61.3% of unseen misbehaviour, whereas those by a state-of-the-art DNN repair technique generalise only to 10.2% and sometimes to none while taking tens of times more than Arachne. We also show that Arachne can address fairness issues by debiasing a gender classification model. Finally, we successfully apply Arachne to a text sentiment model to show that it generalises beyond Convolutional Neural Networks.