pwc-archive/papers-with-abstracts huggingface dataset:
Computational framework for real-time diagnostics and prognostics of aircraft actuation systems
Authors: Pier Carlo Berri, Matteo D. L. Dalla Vedova, Laura Mainini
Date: 2020-09-30 00:00:00
Tasks: Fault Detection, Management

Prognostics and Health Management (PHM) are emerging approaches to product life cycle that will maintain system safety and improve reliability, while reducing operating and maintenance costs. This is particularly relevant for aerospace systems, where high levels of integrity and high performances are required at the same time. We propose a novel strategy for the nearly real-time Fault Detection and Identification (FDI) of a dynamical assembly, and for the estimation of Remaining Useful Life (RUL) of the system. The availability of a timely estimate of the health status of the system will allow for an informed adaptive planning of maintenance and a dynamical reconfiguration of the mission profile, reducing operating costs and improving reliability. This work addresses the three phases of the prognostic flow - namely (1) signal acquisition, (2) Fault Detection and Identification, and (3) Remaining Useful Life estimation - and introduces a computationally efficient procedure suitable for real-time, on-board execution. To achieve this goal, we propose to combine information from physical models of different fidelity with machine learning techniques to obtain efficient representations (surrogate models) suitable for nearly real-time applications. Additionally, we propose an importance sampling strategy and a novel approach to model damage propagation for dynamical systems. The methodology is assessed for the FDI and RUL estimation of an aircraft electromechanical actuator (EMA) for secondary flight controls. The results show that the proposed method allows for a high precision in the evaluation of the system RUL, while outperforming common model-based techniques in terms of computational time.
Dissected 3D CNNs: Temporal Skip Connections for Efficient Online Video Processing
Authors: Okan Köpüklü, Stefan Hörmann, Fabian Herzog, Hakan Cevikalp, Gerhard Rigoll
Date: 2020-09-30 00:00:00
Tasks: Action Classification, Video Recognition

Convolutional Neural Networks with 3D kernels (3D-CNNs) currently achieve state-of-the-art results in video recognition tasks due to their supremacy in extracting spatiotemporal features within video frames. There have been many successful 3D-CNN architectures surpassing the state-of-the-art results successively. However, nearly all of them are designed to operate offline creating several serious handicaps during online operation. Firstly, conventional 3D-CNNs are not dynamic since their output features represent the complete input clip instead of the most recent frame in the clip. Secondly, they are not temporal resolution-preserving due to their inherent temporal downsampling. Lastly, 3D-CNNs are constrained to be used with fixed temporal input size limiting their flexibility. In order to address these drawbacks, we propose dissected 3D-CNNs, where the intermediate volumes of the network are dissected and propagated over depth (time) dimension for future calculations, substantially reducing the number of computations at online operation. For action classification, the dissected version of ResNet models performs 77-90% fewer computations at online operation while achieving ~5% better classification accuracy on the Kinetics-600 dataset than conventional 3D-ResNet models. Moreover, the advantages of dissected 3D-CNNs are demonstrated by deploying our approach onto several vision tasks, which consistently improved the performance.
Adversarial Semi-Supervised Multi-Domain Tracking
Authors: Kourosh Meshgi, Maryam Sadat Mirzaei
Date: 2020-09-30 00:00:00
Tasks: Self-Supervised Learning, Visual Tracking

Neural networks for multi-domain learning empowers an effective combination of information from different domains by sharing and co-learning the parameters. In visual tracking, the emerging features in shared layers of a multi-domain tracker, trained on various sequences, are crucial for tracking in unseen videos. Yet, in a fully shared architecture, some of the emerging features are useful only in a specific domain, reducing the generalization of the learned feature representation. We propose a semi-supervised learning scheme to separate domain-invariant and domain-specific features using adversarial learning, to encourage mutual exclusion between them, and to leverage self-supervised learning for enhancing the shared features using the unlabeled reservoir. By employing these features and training dedicated layers for each sequence, we build a tracker that performs exceptionally on different types of videos.
Concurrent Neural Network : A model of competition between times series
Authors: Rémy Garnier
Date: 2020-09-30 00:00:00
Tasks: Prediction, Time Series, Time Series Analysis

Competition between times series often arises in sales prediction, when similar products are on sale on a marketplace. This article provides a model of the presence of cannibalization between times series. This model creates a "competitiveness" function that depends on external features such as price and margin. It also provides a theoretical guaranty on the error of the model under some reasonable conditions, and implement this model using a neural network to compute this competitiveness function. This implementation outperforms other traditional time series methods and classical neural networks for market share prediction on a real-world data set.
Improving Generalization of Deep Fault Detection Models in the Presence of Mislabeled Data
Authors: Katharina Rombach, Gabriel Michau, Olga Fink
Date: 2020-09-30 00:00:00
Tasks: Data Augmentation, Fault Detection

Mislabeled samples are ubiquitous in real-world datasets as rule-based or expert labeling is usually based on incorrect assumptions or subject to biased opinions. Neural networks can "memorize" these mislabeled samples and, as a result, exhibit poor generalization. This poses a critical issue in fault detection applications, where not only the training but also the validation datasets are prone to contain mislabeled samples. In this work, we propose a novel two-step framework for robust training with label noise. In the first step, we identify outliers (including the mislabeled samples) based on the update in the hypothesis space. In the second step, we propose different approaches to modifying the training data based on the identified outliers and a data augmentation technique. Contrary to previous approaches, we aim at finding a robust solution that is suitable for real-world applications, such as fault detection, where no clean, "noise-free" validation dataset is available. Under an approximate assumption about the upper limit of the label noise, we significantly improve the generalization ability of the model trained under massive label noise.
The Role of Isomorphism Classes in Multi-Relational Datasets
Authors: Vijja Wichitwechkarn, Ben Day, Cristian Bodnar, Matthew Wales, Pietro Liò
Date: 2020-09-30 00:00:00

Multi-interaction systems abound in nature, from colloidal suspensions to gene regulatory circuits. These systems can produce complex dynamics and graph neural networks have been proposed as a method to extract underlying interactions and predict how systems will evolve. The current training and evaluation procedures for these models through the use of synthetic multi-relational datasets however are agnostic to interaction network isomorphism classes, which produce identical dynamics up to initial conditions. We extensively analyse how isomorphism class awareness affects these models, focusing on neural relational inference (NRI) models, which are unique in explicitly inferring interactions to predict dynamics in the unsupervised setting. Specifically, we demonstrate that isomorphism leakage overestimates performance in multi-relational inference and that sampling biases present in the multi-interaction network generation process can impair generalisation. To remedy this, we propose isomorphism-aware synthetic benchmarks for model evaluation. We use these benchmarks to test generalisation abilities and demonstrate the existence of a threshold sampling frequency of isomorphism classes for successful learning. In addition, we demonstrate that isomorphism classes can be utilised through a simple prioritisation scheme to improve model performance, stability during training and reduce training time.
Hidden Markov Models for Pipeline Damage Detection Using Piezoelectric Transducers
Authors: Mingchi Zhang, Xuemin Chen, Wei Li
Date: 2020-09-30 00:00:00

Oil and gas pipeline leakages lead to not only enormous economic loss but also environmental disasters. How to detect the pipeline damages including leakages and cracks has attracted much research attention. One of the promising leakage detection method is to use lead zirconate titanate (PZT) transducers to detect the negative pressure wave when leakage occurs. PZT transducers can generate and detect guided stress waves for crack detection also. However, the negative pressure waves or guided stress waves may not be easily detected with environmental interference, e.g., the oil and gas pipelines in offshore environment. In this paper, a Gaussian mixture model based hidden Markov model (GMM-HMM) method is proposed to detect the pipeline leakage and crack depth in changing environment and time-varying operational conditions. Leakages in different sections or crack depths are considered as different states in hidden Markov models (HMM). Laboratory experiments show that the GMM-HMM method can recognize the crack depth and leakage of pipeline such as whether there is a leakage, where the leakage is.
EWS-GCN: Edge Weight-Shared Graph Convolutional Network for Transactional Banking Data
Authors: Ivan Sukharev, Valentina Shumovskaia, Kirill Fedyanin, Maxim Panov, Dmitry Berestnev
Date: 2020-09-30 00:00:00
Tasks: Graph Neural Network

In this paper, we discuss how modern deep learning approaches can be applied to the credit scoring of bank clients. We show that information about connections between clients based on money transfers between them allows us to significantly improve the quality of credit scoring compared to the approaches using information about the target client solely. As a final solution, we develop a new graph neural network model EWS-GCN that combines ideas of graph convolutional and recurrent neural networks via attention mechanism. The resulting model allows for robust training and efficient processing of large-scale data. We also demonstrate that our model outperforms the state-of-the-art graph neural networks achieving excellent results
Dilated Convolutional Attention Network for Medical Code Assignment from Clinical Text
Authors: Shaoxiong Ji, Erik Cambria, Pekka Marttinen
Date: 2020-09-30 00:00:00
Proceeding: EMNLP (ClinicalNLP) 2020 11

Medical code assignment, which predicts medical codes from clinical texts, is a fundamental task of intelligent medical information systems. The emergence of deep models in natural language processing has boosted the development of automatic assignment methods. However, recent advanced neural architectures with flat convolutions or multi-channel feature concatenation ignore the sequential causal constraint within a text sequence and may not learn meaningful clinical text representations, especially for lengthy clinical notes with long-term sequential dependency. This paper proposes a Dilated Convolutional Attention Network (DCAN), integrating dilated convolutions, residual connections, and label attention, for medical code assignment. It adopts dilated convolutions to capture complex medical patterns with a receptive field which increases exponentially with dilation size. Experiments on a real-world clinical dataset empirically show that our model improves the state of the art.
First-order Optimization for Superquantile-based Supervised Learning
Authors: Yassine Laguel, Jérôme Malick, Zaid Harchaoui
Date: 2020-09-30 00:00:00
Tasks: BIG-bench Machine Learning, regression

Classical supervised learning via empirical risk (or negative log-likelihood) minimization hinges upon the assumption that the testing distribution coincides with the training distribution. This assumption can be challenged in modern applications of machine learning in which learning machines may operate at prediction time with testing data whose distribution departs from the one of the training data. We revisit the superquantile regression method by proposing a first-order optimization algorithm to minimize a superquantile-based learning objective. The proposed algorithm is based on smoothing the superquantile function by infimal convolution. Promising numerical results illustrate the interest of the approach towards safer supervised learning.
Rain-Code Fusion : Code-to-code ConvLSTM Forecasting Spatiotemporal Precipitation
Authors: Takato Yasuno, Akira Ishii, Masazumi Amakata
Date: 2020-09-30 00:00:00
Tasks: Precipitation Forecasting

Recently, flood damage has become a social problem owing to unexperienced weather conditions arising from climate change. An immediate response to heavy rain is important for the mitigation of economic losses and also for rapid recovery. Spatiotemporal precipitation forecasts may enhance the accuracy of dam inflow prediction, more than 6 hours forward for flood damage mitigation. However, the ordinary ConvLSTM has the limitation of predictable range more than 3-timesteps in real-world precipitation forecasting owing to the irreducible bias between target prediction and ground-truth value. This paper proposes a rain-code approach for spatiotemporal precipitation code-to-code forecasting. We propose a novel rainy feature that represents a temporal rainy process using multi-frame fusion for the timestep reduction. We perform rain-code studies with various term ranges based on the standard ConvLSTM. We applied to a dam region within the Japanese rainy term hourly precipitation data, under 2006 to 2019 approximately 127 thousands hours, every year from May to October. We apply the radar analysis hourly data on the central broader region with an area of 136 x 148 km2 . Finally we have provided sensitivity studies between the rain-code size and hourly accuracy within the several forecasting range.
Uncovering Feature Interdependencies in High-Noise Environments with Stepwise Lookahead Decision Forests
Authors: Delilah Donick, Sandro Claudio Lera
Date: 2020-09-30 00:00:00
Tasks: Time Series Analysis

Conventionally, random forests are built from "greedy" decision trees which each consider only one split at a time during their construction. The sub-optimality of greedy implementation has been well-known, yet mainstream adoption of more sophisticated tree building algorithms has been lacking. We examine under what circumstances an implementation of less greedy decision trees actually yields outperformance. To this end, a "stepwise lookahead" variation of the random forest algorithm is presented for its ability to better uncover binary feature interdependencies. In contrast to the greedy approach, the decision trees included in this random forest algorithm, each simultaneously consider three split nodes in tiers of depth two. It is demonstrated on synthetic data and financial price time series that the lookahead version significantly outperforms the greedy one when (a) certain non-linear relationships between feature-pairs are present and (b) if the signal-to-noise ratio is particularly low. A long-short trading strategy for copper futures is then backtested by training both greedy and stepwise lookahead random forests to predict the signs of daily price returns. The resulting superior performance of the lookahead algorithm is at least partially explained by the presence of "XOR-like" relationships between long-term and short-term technical indicators. More generally, across all examined datasets, when no such relationships between features are present, performance across random forests is similar. Given its enhanced ability to understand the feature-interdependencies present in complex systems, this lookahead variation is a useful extension to the toolkit of data scientists, in particular for financial machine learning, where conditions (a) and (b) are typically met.
A robustness measure for singular point and index estimation in discretized orientation and vector fields
Authors: Karl B. Hoffmann, Ivo F. Sbalzarini
Date: 2020-09-30 00:00:00
Tasks: Uncertainty Quantification

The identification of singular points or topological defects in discretized vector fields occurs in diverse areas ranging from the polarization of the cosmic microwave background to liquid crystals to fingerprint recognition and bio-medical imaging. Due to their discrete nature, defects and their topological charge cannot depend continuously on each single vector, but they discontinuously change as soon as a vector changes by more than a threshold. Considering this threshold of admissible change at the level of vectors, we develop a robustness measure for discrete defect estimators. Here, we compare different template paths for defect estimation in discretized vector or orientation fields. Sampling prototypical vector field patterns around defects shows that the robustness increases with the length of template path, but less so in the presence of noise on the vectors. We therefore find an optimal trade-off between resolution and robustness against noise for relatively small templates, except for the "single pixel" defect analysis, which cannot exclude zero robustness. The presented robustness measure paves the way for uncertainty quantification of defects in discretized vector fields.
Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network
Authors: Sijin Kim, Namhyuk Ahn, Kyung-Ah Sohn
Date: 2020-09-30 00:00:00
Tasks: Mixture-of-Experts, Multi-Task Learning

In recent years, deep learning-based methods have been successfully applied to the image distortion restoration tasks. However, scenarios that assume a single distortion only may not be suitable for many real-world applications. To deal with such cases, some studies have proposed sequentially combined distortions datasets. Viewing in a different point of combining, we introduce a spatially-heterogeneous distortion dataset in which multiple corruptions are applied to the different locations of each image. In addition, we also propose a mixture of experts network to effectively restore a multi-distortion image. Motivated by the multi-task learning, we design our network to have multiple paths that learn both common and distortion-specific representations. Our model is effective for restoring real-world distortions and we experimentally verify that our method outperforms other models designed to manage both single distortion and multiple distortions.
Learning Object Detection from Captions via Textual Scene Attributes
Authors: Achiya Jerbi, Roei Herzig, Jonathan Berant, Gal Chechik, Amir Globerson
Date: 2020-09-30 00:00:00
Tasks: Image Captioning, Object, object-detection, Object Detection

Object detection is a fundamental task in computer vision, requiring large annotated datasets that are difficult to collect, as annotators need to label objects and their bounding boxes. Thus, it is a significant challenge to use cheaper forms of supervision effectively. Recent work has begun to explore image captions as a source for weak supervision, but to date, in the context of object detection, captions have only been used to infer the categories of the objects in the image. In this work, we argue that captions contain much richer information about the image, including attributes of objects and their relations. Namely, the text represents a scene of the image, as described recently in the literature. We present a method that uses the attributes in this "textual scene graph" to train object detectors. We empirically demonstrate that the resulting model achieves state-of-the-art results on several challenging object detection datasets, outperforming recent approaches.
One Reflection Suffice
Authors: Alexander Mathiasen, Frederik Hvilshøj
Date: 2020-09-30 00:00:00
Tasks: GPU

Orthogonal weight matrices are used in many areas of deep learning. Much previous work attempt to alleviate the additional computational resources it requires to constrain weight matrices to be orthogonal. One popular approach utilizes *many* Householder reflections. The only practical drawback is that many reflections cause low GPU utilization. We mitigate this final drawback by proving that *one* reflection is sufficient, if the reflection is computed by an auxiliary neural network.
Wasserstein Distributionally Robust Inverse Multiobjective Optimization
Authors: Chaosheng Dong, Bo Zeng
Date: 2020-09-30 00:00:00
Tasks: Decision Making, Multiobjective Optimization, Portfolio Optimization

Inverse multiobjective optimization provides a general framework for the unsupervised learning task of inferring parameters of a multiobjective decision making problem (DMP), based on a set of observed decisions from the human expert. However, the performance of this framework relies critically on the availability of an accurate DMP, sufficient decisions of high quality, and a parameter space that contains enough information about the DMP. To hedge against the uncertainties in the hypothetical DMP, the data, and the parameter space, we investigate in this paper the distributionally robust approach for inverse multiobjective optimization. Specifically, we leverage the Wasserstein metric to construct a ball centered at the empirical distribution of these decisions. We then formulate a Wasserstein distributionally robust inverse multiobjective optimization problem (WRO-IMOP) that minimizes a worst-case expected loss function, where the worst case is taken over all distributions in the Wasserstein ball. We show that the excess risk of the WRO-IMOP estimator has a sub-linear convergence rate. Furthermore, we propose the semi-infinite reformulations of the WRO-IMOP and develop a cutting-plane algorithm that converges to an approximate solution in finite iterations. Finally, we demonstrate the effectiveness of our method on both a synthetic multiobjective quadratic program and a real world portfolio optimization problem.
FAN: Frequency Aggregation Network for Real Image Super-resolution
Authors: Yingxue Pang, Xin Li, Xin Jin, Yaojun Wu, Jianzhao Liu, Sen Liu, Zhibo Chen
Date: 2020-09-30 00:00:00
Tasks: Image Super-Resolution, SSIM, Super-Resolution

Single image super-resolution (SISR) aims to recover the high-resolution (HR) image from its low-resolution (LR) input image. With the development of deep learning, SISR has achieved great progress. However, It is still a challenge to restore the real-world LR image with complicated authentic degradations. Therefore, we propose FAN, a frequency aggregation network, to address the real-world image super-resolu-tion problem. Specifically, we extract different frequencies of the LR image and pass them to a channel attention-grouped residual dense network (CA-GRDB) individually to output corresponding feature maps. And then aggregating these residual dense feature maps adaptively to recover the HR image with enhanced details and textures. We conduct extensive experiments quantitatively and qualitatively to verify that our FAN performs well on the real image super-resolution task of AIM 2020 challenge. According to the released final results, our team SR-IM achieves the fourth place on the X4 track with PSNR of 31.1735 and SSIM of 0.8728.