pwc-archive/papers-with-abstracts huggingface dataset:
ALIME: Autoencoder Based Approach for Local Interpretability
Authors: Sharath M. Shankaranarayana, Davor Runje
Date: 2019-09-04 00:00:00
Tasks: Deep Learning

Machine learning and especially deep learning have garneredtremendous popularity in recent years due to their increased performanceover other methods. The availability of large amount of data has aidedin the progress of deep learning. Nevertheless, deep learning models areopaque and often seen as black boxes. Thus, there is an inherent need tomake the models interpretable, especially so in the medical domain. Inthis work, we propose a locally interpretable method, which is inspiredby one of the recent tools that has gained a lot of interest, called localinterpretable model-agnostic explanations (LIME). LIME generates singleinstance level explanation by artificially generating a dataset aroundthe instance (by randomly sampling and using perturbations) and thentraining a local linear interpretable model. One of the major issues inLIME is the instability in the generated explanation, which is caused dueto the randomly generated dataset. Another issue in these kind of localinterpretable models is the local fidelity. We propose novel modificationsto LIME by employing an autoencoder, which serves as a better weightingfunction for the local model. We perform extensive comparisons withdifferent datasets and show that our proposed method results in bothimproved stability, as well as local fidelity.
Random Error Sampling-based Recurrent Neural Network Architecture Optimization
Authors: Andrés Camero, Jamal Toutouh, Enrique Alba
Date: 2019-09-04 00:00:00
Tasks: Hyperparameter Optimization

Recurrent neural networks are good at solving prediction problems. However, finding a network that suits a problem is quite hard because their performance is strongly affected by their architecture configuration. Automatic architecture optimization methods help to find the most suitable design, but they are not extensively adopted because of their high computational cost. In this work, we introduce the Random Error Sampling-based Neuroevolution (RESN), an evolutionary algorithm that uses the mean absolute error random sampling, a training-free approach to predict the expected performance of an artificial neural network, to optimize the architecture of a network. We empirically validate our proposal on three prediction problems, and compare our technique to training-based architecture optimization techniques and to neuroevolutionary approaches. Our findings show that we can achieve state-of-the-art error performance and that we reduce by half the time needed to perform the optimization.
Artificial Neural Networks and Adaptive Neuro-fuzzy Models for Prediction of Remaining Useful Life
Authors: Razieh Tavakoli, Mohammad Najafi, Ali Sharifara
Date: 2019-08-28 00:00:00

The U.S. water distribution system contains thousands of miles of pipes constructed from different materials, and of various sizes, and age. These pipes suffer from physical, environmental, structural and operational stresses, causing deterioration which eventually leads to their failure. Pipe deterioration results in increased break rates, reduced hydraulic capacity, and detrimental impacts on water quality. Therefore, it is crucial to use accurate models to forecast deterioration rates along with estimating the remaining useful life of the pipes to implement essential interference plans in order to prevent catastrophic failures. This paper discusses a computational model that forecasts the RUL of water pipes by applying Artificial Neural Networks (ANNs) as well as Adaptive Neural Fuzzy Inference System (ANFIS). These models are trained and tested acquired field data to identify the significant parameters that impact the prediction of RUL. It is concluded that, on average, with approximately 10\% of wall thickness loss in existing cast iron, ductile iron, asbestos-cement, and steel water pipes, the reduction of the remaining useful life is approximately 50%
Data-driven simulation for general purpose multibody dynamics using deep neural networks
Authors: Hee-Sun Choi, Junmo An, Jin-Gyun Kim, Jae-Yoon Jung, Juhwan Choi, Grzegorz Orzechowski, Aki Mikkola, Jin Hwan Choi
Date: 2019-09-02 00:00:00
Tasks: Motion Estimation

In this paper, a machine learning-based simulation framework of general-purpose multibody dynamics is introduced. The aim of the framework is to generate a well-trained meta-model of multibody dynamics (MBD) systems. To this end, deep neural network (DNN) is employed to the framework so as to construct data-based meta-model representing multibody systems. Constructing well-defined training data set with time variable is essential to get accurate and reliable motion data such as displacement, velocity, acceleration, and forces. As a result of the introduced approach, the meta-model provides motion estimation of system dynamics without solving the analytical equations of motion. The performance of the proposed DNN meta-modeling was evaluated to represent several MBD systems.
Riemannian batch normalization for SPD neural networks
Authors: Daniel Brooks, Olivier Schwander, Frederic Barbaresco, Jean-Yves Schneider, Matthieu Cord
Date: 2019-09-03 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Action Recognition

Covariance matrices have attracted attention for machine learning applications due to their capacity to capture interesting structure in the data. The main challenge is that one needs to take into account the particular geometry of the Riemannian manifold of symmetric positive definite (SPD) matrices they belong to. In the context of deep networks, several architectures for these matrices have recently been proposed. In our article, we introduce a Riemannian batch normalization (batchnorm) algorithm, which generalizes the one used in Euclidean nets. This novel layer makes use of geometric operations on the manifold, notably the Riemannian barycenter, parallel transport and non-linear structured matrix transformations. We derive a new manifold-constrained gradient descent algorithm working in the space of SPD matrices, allowing to learn the batchnorm layer. We validate our proposed approach with experiments in three different contexts on diverse data types: a drone recognition dataset from radar observations, and on emotion and action recognition datasets from video and motion capture data. Experiments show that the Riemannian batchnorm systematically gives better classification performance compared with leading methods and a remarkable robustness to lack of data.
On perfectness in Gaussian graphical models
Authors: Arash A. Amini, Bryon Aragam, Qing Zhou
Date: 2019-09-03 00:00:00

Knowing when a graphical model is perfect to a distribution is essential in order to relate separation in the graph to conditional independence in the distribution, and this is particularly important when performing inference from data. When the model is perfect, there is a one-to-one correspondence between conditional independence statements in the distribution and separation statements in the graph. Previous work has shown that almost all models based on linear directed acyclic graphs as well as Gaussian chain graphs are perfect, the latter of which subsumes Gaussian graphical models (i.e., the undirected Gaussian models) as a special case. However, the complexity of chain graph models leads to a proof of this result which is indirect and mired by the complications of parameterizing this general class. In this paper, we directly approach the problem of perfectness for the Gaussian graphical models, and provide a new proof, via a more transparent parametrization, that almost all such models are perfect. Our approach is based on, and substantially extends, a construction of Ln\v{e}ni\v{c}ka and Mat\'u\v{s} showing the existence of a perfect Gaussian distribution for any graph.
Evaluating Conformance Measures in Process Mining using Conformance Propositions (Extended version)
Authors: Anja F. Syring, Niek Tax, Wil M. P. van der Aalst
Date: 2019-08-30 00:00:00

Process mining sheds new light on the relationship between process models and real-life processes. Process discovery can be used to learn process models from event logs. Conformance checking is concerned with quantifying the quality of a business process model in relation to event data that was logged during the execution of the business process. There exist different categories of conformance measures. Recall, also called fitness, is concerned with quantifying how much of the behavior that was observed in the event log fits the process model. Precision is concerned with quantifying how much behavior a process model allows for that was never observed in the event log. Generalization is concerned with quantifying how well a process model generalizes to behavior that is possible in the business process but was never observed in the event log. Many recall, precision, and generalization measures have been developed throughout the years, but they are often defined in an ad-hoc manner without formally defining the desired properties up front. To address these problems, we formulate 21 conformance propositions and we use these propositions to evaluate current and existing conformance measures. The goal is to trigger a discussion by clearly formulating the challenges and requirements (rather than proposing new measures). Additionally, this paper serves as an overview of the conformance checking measures that are available in the process mining area.
High-Fidelity State-of-Charge Estimation of Li-Ion Batteries Using Machine Learning
Authors: Weizhong Wang, Nicholas W. Brady, Chenyao Liao, Youssef A. Fahmy, Ephrem Chemali, Alan C. West, Matthias Preindl
Date: 2019-08-30 00:00:00
Tasks: BIG-bench Machine Learning

This paper proposes a way to augment the existing machine learning algorithm applied to state-of-charge estimation by introducing a form of pulse injection to the running battery cells. It is believed that the information contained in the pulse responses can be interpreted by a machine learning algorithm whereas other techniques are difficult to decode due to the nonlinearity. The sensitivity analysis of the amplitude of the current pulse is given through simulation, allowing the researchers to select the appropriate current level with respect to the desired accuracy improvement. A multi-layer feedforward neural networks is trained to acquire the nonlinear relationship between the pulse train and the ground-truth SoC. The experimental data is trained and the results are shown to be promising with less than 2\% SoC estimation error using layer sizes in the range of 10 - 10,000 trained in 0 - 1 million epochs. The testing procedure specifically designed for the proposed technique is explained and provided. The implementation of the proposed strategy is also discussed. The detailed system layout to perform the augmented SoC estimation integrated in the existing active balancing hardware has also been given.
Explanation based Handwriting Verification
Authors: Mihir Chauhan, Mohammad Abuzar Shaikh, Sargur N. Srihari
Date: 2019-08-14 00:00:00
Tasks: Explainable Artificial Intelligence (XAI), Face Verification, Handwriting Verification

Deep learning system have drawback that their output is not accompanied with ex-planation. In a domain such as forensic handwriting verification it is essential to provideexplanation to jurors. The goal of handwriting verification is to find a measure of confi-dence whether the given handwritten samples are written by the same or different writer.We propose a method to generate explanations for the confidence provided by convolu-tional neural network (CNN) which maps the input image to 15 annotations (features)provided by experts. Our system comprises of: (1) Feature learning network (FLN),a differentiable system, (2) Inference module for providing explanations. Furthermore,inference module provides two types of explanations: (a) Based on cosine similaritybetween categorical probabilities of each feature, (b) Based on Log-Likelihood Ratio(LLR) using directed probabilistic graphical model. We perform experiments using acombination of feature learning network (FLN) and each inference module. We evaluateour system using XAI-AND dataset, containing 13700 handwritten samples and 15 cor-responding expert examined features for each sample. The dataset is released for publicuse and the methods can be extended to provide explanations on other verification taskslike face verification and bio-medical comparison. This dataset can serve as the basis and benchmark for future research in explanation based handwriting verification. The code is available on github.
Functional Sequential Treatment Allocation
Authors: Anders Bredahl Kock, David Preinerstorfer, Bezirgen Veliyev
Date: 2018-12-21 00:00:00

Consider a setting in which a policy maker assigns subjects to treatments, observing each outcome before the next subject arrives. Initially, it is unknown which treatment is best. However, the sequential nature of the problem permits learning about the effectiveness of the treatments, which we measure by a functional of the associated outcome distributions, for example an inequality, welfare or poverty measure. In the present article, we evaluate assignment policies according to their regret, that is, the sum of all losses incurred due to assigning subjects to suboptimal treatments. We first study explore-then-commit (ETC) policies. These are policies, where one initially explores which treatment is of the highest quality, typically through a randomized controlled trial, and subsequently fully commits to the "inferred superior" (but potentially suboptimal) treatment. Then, we introduce and study the Functional Upper Confidence Bound (F-UCB) policy, which interweaves exploration and exploitation and is thus not of the ETC type. Our results show, in particular, that the F-UCB policy (i) performs much better than any ETC policy, and (ii) is near minimax optimal. We also show that a suitably adapted F-UCB policy is near minimax optimal under minimal assumptions when covariate information is available.
Do Cross Modal Systems Leverage Semantic Relationships?
Authors: Shah Nawaz, Muhammad Kamran Janjua, Ignazio Gallo, Arif Mahmood, Alessandro Calefati, Faisal Shafait
Date: 2019-09-03 00:00:00
Tasks: Cross-Modal Retrieval, Retrieval, Semantic Similarity, Semantic Textual Similarity

Current cross-modal retrieval systems are evaluated using R@K measure which does not leverage semantic relationships rather strictly follows the manually marked image text query pairs. Therefore, current systems do not generalize well for the unseen data in the wild. To handle this, we propose a new measure, SemanticMap, to evaluate the performance of cross-modal systems. Our proposed measure evaluates the semantic similarity between the image and text representations in the latent embedding space. We also propose a novel cross-modal retrieval system using a single stream network for bidirectional retrieval. The proposed system is based on a deep neural network trained using extended center loss, minimizing the distance of image and text descriptions in the latent space from the class centers. In our system, the text descriptions are also encoded as images which enabled us to use a single stream network for both text and images. To the best of our knowledge, our work is the first of its kind in terms of employing a single stream network for cross-modal retrieval systems. The proposed system is evaluated on two publicly available datasets including MSCOCO and Flickr30K and has shown comparable results to the current state-of-the-art methods.
ApproxNet: Content and Contention-Aware Video Analytics System for Embedded Clients
Authors: Ran Xu, Rakesh Kumar, Pengcheng Wang, Peter Bai, Ganga Meghanath, Somali Chaterji, Subrata Mitra, Saurabh Bagchi
Date: 2019-08-28 00:00:00
Tasks: Object Detection

Videos take a lot of time to transport over the network, hence running analytics on the live video on embedded or mobile devices has become an important system driver. Considering that such devices, e.g., surveillance cameras or AR/VR gadgets, are resource constrained, creating lightweight deep neural networks (DNNs) for embedded devices is crucial. None of the current approximation techniques for object classification DNNs can adapt to changing runtime conditions, e.g., changes in resource availability on the device, the content characteristics, or requirements from the user. In this paper, we introduce ApproxNet, a video object classification system for embedded or mobile clients. It enables novel dynamic approximation techniques to achieve desired inference latency and accuracy trade-off under changing runtime conditions. It achieves this by enabling two approximation knobs within a single DNN model, rather than creating and maintaining an ensemble of models (e.g., MCDNN [MobiSys-16]. We show that ApproxNet can adapt seamlessly at runtime to these changes, provides low and stable latency for the image and video frame classification problems, and show the improvement in accuracy and latency over ResNet [CVPR-16], MCDNN [MobiSys-16], MobileNets [Google-17], NestDNN [MobiCom-18], and MSDNet [ICLR-18].
Deeper Profiles and Cascaded Recurrent and Convolutional Neural Networks for state-of-the-art Protein Secondary Structure Prediction
Authors: Mirko Torrisi, Manaz Kaleel, Gianluca Pollastri
Date: 2019-10-12 00:00:00
Proceeding: Scientific Reports 2019 10
Tasks: Protein Secondary Structure Prediction

Protein Secondary Structure prediction has been a central topic of research in Bioinformatics for decades. In spite of this, even the most sophisticated ab initio SS predictors are not able to reach the theoretical limit of three-state prediction accuracy (88–90%), while only a few predict more than the 3 traditional Helix, Strand and Coil classes. In this study we present tests on different models trained both on single sequence and evolutionary profile-based inputs and develop a new state-of-the-art system with Porter 5. Porter 5 is composed of ensembles of cascaded Bidirectional Recurrent Neural Networks and Convolutional Neural Networks, incorporates new input encoding techniques and is trained on a large set of protein structures. Porter 5 achieves 84% accuracy (81% SOV) when tested on 3 classes and 73% accuracy (70% SOV) on 8 classes on a large independent set. In our tests Porter 5 is 2% more accurate than its previous version and outperforms or matches the most recent predictors of secondary structure we tested. When Porter 5 is retrained on SCOPe based sets that eliminate homology between training/testing samples we obtain similar results. Porter is available as a web server and standalone program at http://distilldeep.ucd.ie/porter/ alongside all the datasets and alignments.
Investigating Rumor News Using Agreement-Aware Search
Authors: Jingbo Shang, Tianhang Sun, Jiaming Shen, Xingbang Liu, Anja Gruenheid, Flip Korn, Adam Lelkes, Cong Yu, Jiawei Han
Date: 2018-02-21 00:00:00
Tasks: Articles

Recent years have witnessed a widespread increase of rumor news generated by humans and machines. Therefore, tools for investigating rumor news have become an urgent necessity. One useful function of such tools is to see ways a specific topic or event is represented by presenting different points of view from multiple sources. In this paper, we propose Maester, a novel agreement-aware search framework for investigating rumor news. Given an investigative question, Maester will retrieve related articles to that question, assign and display top articles from agree, disagree, and discuss categories to users. Splitting the results into these three categories provides the user a holistic view towards the investigative question. We build Maester based on the following two key observations: (1) relatedness can commonly be determined by keywords and entities occurring in both questions and articles, and (2) the level of agreement between the investigative question and the related news article can often be decided by a few key sentences. Accordingly, we use gradient boosting tree models with keyword/entity matching features for relatedness detection, and leverage recurrent neural network to infer the level of agreement. Our experiments on the Fake News Challenge (FNC) dataset demonstrate up to an order of magnitude improvement of Maester over the original FNC winning solution, for agreement-aware search.
Rates of Convergence for Large-scale Nearest Neighbor Classification
Authors: Xingye Qiao, Jiexin Duan, Guang Cheng
Date: 2019-09-03 00:00:00
Proceeding: NeurIPS 2019 12
Tasks: Classification, General Classification

Nearest neighbor is a popular class of classification methods with many desirable properties. For a large data set which cannot be loaded into the memory of a single machine due to computation, communication, privacy, or ownership limitations, we consider the divide and conquer scheme: the entire data set is divided into small subsamples, on which nearest neighbor predictions are made, and then a final decision is reached by aggregating the predictions on subsamples by majority voting. We name this method the big Nearest Neighbor (bigNN) classifier, and provide its rates of convergence under minimal assumptions, in terms of both the excess risk and the classification instability, which are proven to be the same rates as the oracle nearest neighbor classifier and cannot be improved. To significantly reduce the prediction time that is required for achieving the optimal rate, we also consider the pre-training acceleration technique applied to the bigNN method, with proven convergence rate. We find that in the distributed setting, the optimal choice of the neighbor $k$ should scale with both the total sample size and the number of partitions, and there is a theoretical upper limit for the latter. Numerical studies have verified the theoretical findings.
Parsing Natural Scenes and Natural Language with Recursive Neural Networks
Authors: Richard Socher，Cliff Chiung-Yu Lin，Andrew Y. Ng，Christopher D. Manning
Date: 2011-06-01 00:00:00
Proceeding: Proceedings of the 26th International Conference on Machine Learning (ICML) 2011 2011 6
Tasks: General Classification, Scene Classification, Scene Segmentation, Scene Understanding, Segmentation, Sentence

Recursive structure is commonly found in the inputs of different modalities such as natural scene images or natural language sentences.Discovering this recursive structure helps us to not only identify the units that an image or sentence contains but also how they interact to form a whole. We introduce a max-margin structure prediction architecture based on recursive neural networks that can successfully recover such structure both in complex scene images as well as sentences. The same algorithm can be used both to provide a competitive syntactic parser for natural language sentences from the Penn Treebank and to out-perform alternative approaches for semantic scene segmentation, annotation and classification. For segmentation and annotation our algorithm obtains a new level of state-of-the-art performance on the Stanford background dataset (78.1%). The features from the im-age parse tree outperform Gist descriptors forscene classification by 4%.