pwc-archive/papers-with-abstracts huggingface dataset:
Self-Supervised Encoder for Fault Prediction in Electrochemical Cells
Authors: Daniel Buades Marcos, Soumaya Yacout, Said Berriah
Date: 2020-06-05 00:00:00
Tasks: Decoder, Prediction

Predicting faults before they occur helps to avoid potential safety hazards. Furthermore, planning the required maintenance actions in advance reduces operation costs. In this article, the focus is on electrochemical cells. In order to predict a cell's fault, the typical approach is to estimate the expected voltage that a healthy cell would present and compare it with the cell's measured voltage in real-time. This approach is possible because, when a fault is about to happen, the cell's measured voltage differs from the one expected for the same operating conditions. However, estimating the expected voltage is challenging, as the voltage of a healthy cell is also affected by its degradation -- an unknown parameter. Expert-defined parametric models are currently used for this estimation task. Instead, we propose the use of a neural network model based on an encoder-decoder architecture. The network receives the operating conditions as input. The encoder's task is to find a faithful representation of the cell's degradation and to pass it to the decoder, which in turn predicts the expected cell's voltage. As no labeled degradation data is given to the network, we consider our approach to be a self-supervised encoder. Results show that we were able to predict the voltage of multiple cells while diminishing the prediction error that was obtained by the parametric models by 53%. This improvement enabled our network to predict a fault 31 hours before it happened, a 64% increase in reaction time compared to the parametric model. Moreover, the output of the encoder can be plotted, adding interpretability to the neural network model.
Neural Named Entity Recognition for Kazakh
Authors: Gulmira Tolegen, Alymzhan Toleu, Orken Mamyrbayev, Rustam Mussabayev
Date: 2020-07-17 00:00:00
Tasks: named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, TAG

We present several neural networks to address the task of named entity recognition for morphologically complex languages (MCL). Kazakh is a morphologically complex language in which each root/stem can produce hundreds or thousands of variant word forms. This nature of the language could lead to a serious data sparsity problem, which may prevent the deep learning models from being well trained for under-resourced MCLs. In order to model the MCLs' words effectively, we introduce root and entity tag embedding plus tensor layer to the neural networks. The effects of those are significant for improving NER model performance of MCLs. The proposed models outperform state-of-the-art including character-based approaches, and can be potentially applied to other morphologically complex languages.
Identification, Tracking and Impact: Understanding the trade secret of catchphrases
Authors: Jagriti Jalal, Mayank Singh, Arindam Pal, Lipika Dey, Animesh Mukherjee
Date: 2020-07-20 00:00:00

Understanding the topical evolution in industrial innovation is a challenging problem. With the advancement in the digital repositories in the form of patent documents, it is becoming increasingly more feasible to understand the innovation secrets -- "catchphrases" of organizations. However, searching and understanding this enormous textual information is a natural bottleneck. In this paper, we propose an unsupervised method for the extraction of catchphrases from the abstracts of patents granted by the U.S. Patent and Trademark Office over the years. Our proposed system achieves substantial improvement, both in terms of precision and recall, against state-of-the-art techniques. As a second objective, we conduct an extensive empirical study to understand the temporal evolution of the catchphrases across various organizations. We also show how the overall innovation evolution in the form of introduction of newer catchphrases in an organization's patents correlates with the future citations received by the patents filed by that organization. Our code and data sets will be placed in the public domain soon.
Coupling Learning of Complex Interactions
Authors: Longbing Cao
Date: 2020-07-01 00:00:00
Tasks: Clustering, Recommendation Systems

Complex applications such as big data analytics involve different forms of coupling relationships that reflect interactions between factors related to technical, business (domain-specific) and environmental (including socio-cultural and economic) aspects. There are diverse forms of couplings embedded in poor-structured and ill-structured data. Such couplings are ubiquitous, implicit and/or explicit, objective and/or subjective, heterogeneous and/or homogeneous, presenting complexities to existing learning systems in statistics, mathematics and computer sciences, such as typical dependency, association and correlation relationships. Modeling and learning such couplings thus is fundamental but challenging. This paper discusses the concept of coupling learning, focusing on the involvement of coupling relationships in learning systems. Coupling learning has great potential for building a deep understanding of the essence of business problems and handling challenges that have not been addressed well by existing learning theories and tools. This argument is verified by several case studies on coupling learning, including handling coupling in recommender systems, incorporating couplings into coupled clustering, coupling document clustering, coupled recommender algorithms and coupled behavior analysis for groups.
A Novel adaptive optimization of Dual-Tree Complex Wavelet Transform for Medical Image Fusion
Authors: T. Deepika, G. Karpaga Kannan
Date: 2020-07-22 00:00:00
Tasks: SSIM

In recent years, many research achievements are made in the medical image fusion field. Fusion is basically extraction of best of inputs and conveying it to the output. Medical Image fusion means that several of various modality image information is comprehended together to form one image to express its information. The aim of image fusion is to integrate complementary and redundant information. In this paper, a multimodal image fusion algorithm based on the dual-tree complex wavelet transform (DT-CWT) and adaptive particle swarm optimization (APSO) is proposed. Fusion is achieved through the formation of a fused pyramid using the DTCWT coefficients from the decomposed pyramids of the source images. The coefficients are fused by the weighted average method based on pixels, and the weights are estimated by the APSO to gain optimal fused images. The fused image is obtained through conventional inverse dual-tree complex wavelet transform reconstruction process. Experiment results show that the proposed method based on adaptive particle swarm optimization algorithm is remarkably better than the method based on particle swarm optimization. The resulting fused images are compared visually and through benchmarks such as Entropy (E), Peak Signal to Noise Ratio, (PSNR), Root Mean Square Error (RMSE), Standard deviation (SD) and Structure Similarity Index Metric (SSIM) computations.
SparseTrain: Exploiting Dataflow Sparsity for Efficient Convolutional Neural Networks Training
Authors: Pengcheng Dai, Jianlei Yang, Xucheng Ye, Xingzhou Cheng, Junyu Luo, Linghao Song, Yiran Chen, Weisheng Zhao
Date: 2020-07-21 00:00:00

Training Convolutional Neural Networks (CNNs) usually requires a large number of computational resources. In this paper, \textit{SparseTrain} is proposed to accelerate CNN training by fully exploiting the sparsity. It mainly involves three levels of innovations: activation gradients pruning algorithm, sparse training dataflow, and accelerator architecture. By applying a stochastic pruning algorithm on each layer, the sparsity of back-propagation gradients can be increased dramatically without degrading training accuracy and convergence rate. Moreover, to utilize both \textit{natural sparsity} (resulted from ReLU or Pooling layers) and \textit{artificial sparsity} (brought by pruning algorithm), a sparse-aware architecture is proposed for training acceleration. This architecture supports forward and back-propagation of CNN by adopting 1-Dimensional convolution dataflow. We have built %a simple compiler to map CNNs topology onto \textit{SparseTrain}, and a cycle-accurate architecture simulator to evaluate the performance and efficiency based on the synthesized design with $14nm$ FinFET technologies. Evaluation results on AlexNet/ResNet show that \textit{SparseTrain} could achieve about $2.7 \times$ speedup and $2.2 \times$ energy efficiency improvement on average compared with the original training process.
Attention-based Graph ResNet for Motor Intent Detection from Raw EEG signals
Authors: Shuyue Jia, Yimin Hou, Yan Shi, Yang Li
Date: 2020-06-25 00:00:00
Tasks: EEG, Electroencephalogram (EEG), Intent Detection, Motor Imagery, Seizure prediction

In previous studies, decoding electroencephalography (EEG) signals has not considered the topological relationship of EEG electrodes. However, the latest neuroscience has suggested brain network connectivity. Thus, the exhibited interaction between EEG channels might not be appropriately measured via Euclidean distance. To fill the gap, an attention-based graph residual network, a novel structure of Graph Convolutional Neural Network (GCN), was presented to detect human motor intents from raw EEG signals, where the topological structure of EEG electrodes was built as a graph. Meanwhile, deep residual learning with a full-attention architecture was introduced to address the degradation problem concerning deeper networks in raw EEG motor imagery (MI) data. Individual variability, the critical and longstanding challenge underlying EEG signals, has been successfully handled with the state-of-the-art performance, 98.08% accuracy at the subject level, 94.28% for 20 subjects. Numerical results were promising that the implementation of the graph-structured topology was superior to decode raw EEG data. The innovative deep learning approach was expected to entail a universal method towards both neuroscience research and real-world EEG-based practical applications, e.g., seizure prediction.
Hardware Implementation of Hyperbolic Tangent Function using Catmull-Rom Spline Interpolation
Authors: Mahesh Chandra
Date: 2020-07-13 00:00:00
Tasks: Object Recognition, speech-recognition, Speech Recognition

Deep neural networks yield the state of the art results in many computer vision and human machine interface tasks such as object recognition, speech recognition etc. Since, these networks are computationally expensive, customized accelerators are designed for achieving the required performance at lower cost and power. One of the key building blocks of these neural networks is non-linear activation function such as sigmoid, hyperbolic tangent (tanh), and ReLU. A low complexity accurate hardware implementation of the activation function is required to meet the performance and area targets of the neural network accelerators. This paper presents an implementation of tanh function using the Catmull-Rom spline interpolation. State of the art results are achieved using this method with comparatively smaller logic area.
Cloud Detection through Wavelet Transforms in Machine Learning and Deep Learning
Authors: Philippe Reiter
Date: 2020-07-10 00:00:00
Tasks: BIG-bench Machine Learning, Cloud Detection, object-detection, Object Detection

Cloud detection is a specialized application of image recognition and object detection using remotely sensed data. The task presents a number of challenges, including analyzing images obtained in visible, infrared and multi-spectral frequencies, usually without ground truth data for comparison. Moreover, machine learning and deep learning (MLDL) algorithms applied to this task are required to be computationally efficient, as they are typically deployed in low-power devices and called to operate in real-time. This paper explains Wavelet Transform (WT) theory, comparing it to more widely used image and signal processing transforms, and explores the use of WT as a powerful signal compressor and feature extractor for MLDL classifiers.
MultiWOZ 2.2 : A Dialogue Dataset with Additional Annotation Corrections and State Tracking Baselines
Authors: Xiaoxue Zang, Abhinav Rastogi, Srinivas Sunkara, Raghav Gupta, Jian-Guo Zhang, Jindong Chen
Date: 2020-07-10 00:00:00
Proceeding: WS 2020 7
Tasks: Dialogue State Tracking

MultiWOZ is a well-known task-oriented dialogue dataset containing over 10,000 annotated dialogues spanning 8 domains. It is extensively used as a benchmark for dialogue state tracking. However, recent works have reported presence of substantial noise in the dialogue state annotations. MultiWOZ 2.1 identified and fixed many of these erroneous annotations and user utterances, resulting in an improved version of this dataset. This work introduces MultiWOZ 2.2, which is a yet another improved version of this dataset. Firstly, we identify and fix dialogue state annotation errors across 17.3% of the utterances on top of MultiWOZ 2.1. Secondly, we redefine the ontology by disallowing vocabularies of slots with a large number of possible values (e.g., restaurant name, time of booking). In addition, we introduce slot span annotations for these slots to standardize them across recent models, which previously used custom string matching heuristics to generate them. We also benchmark a few state of the art dialogue state tracking models on the corrected dataset to facilitate comparison for future work. In the end, we discuss best practices for dialogue data collection that can help avoid annotation errors.
Towards an ontology of HTTP interactions
Authors: Mathieu Lirzin, Béatrice Markhoff
Date: 2020-07-20 00:00:00

Enterprise information systems have adopted Web-based foundations for exchanges between heterogeneous programmes. These programs provide and consume via Web APIs some resources identified by URIs, whose representations are transmitted via HTTP. Furthermore HTTP remains at the heart of all Web developments (Semantic Web, linked data, IoT...). Thus, situations where a program must be able to reason about HTTP interactions (request-response) are multiplying. This requires an explicit formal specification of a shared conceptualization of those interactions. A proposal for an RDF vocabulary exists, developed with a view to carrying out web application conformity tests and record the tests outputs. This vocabulary has already been reused. In this paper we propose to adapt and extend it for making it more reusable.
Intelligent Reputation System for Safety Messages in VANET
Authors: Ghassan Samara
Date: 2020-06-12 00:00:00
Tasks: Decision Making, Position

Nowadays, Vehicle Ad - hoc Nets (VANET) applications have become very important in our lives because VANET provides drivers with safety messages, warnings, and instructions to ensure drivers have a safe and enjoyable journey. VANET Security is one of the hottest topics in computer networks research, Falsifying VANET system information violates VANET safety objectives and may lead to hazardous situations and loss of life. In this paper, an Intelligent Reputation System (IRS) aims to identify attacking vehicles will be proposed; the proposed system will rely on opinion generation, trust value collection, traffic analysis, position based, data collection, and intelligent decision making by utilizing the multi-parameter Greedy Best First algorithm. The results of this research will enhance VANET's safety level and will facilitate the identification of misbehaving vehicles and their messages. The results of the proposed system have also proven to be superior to other reputational systems.
Post-Workshop Report on Science meets Engineering in Deep Learning, NeurIPS 2019, Vancouver
Authors: Levent Sagun, Caglar Gulcehre, Adriana Romero, Negar Rostamzadeh, Stefano Sarao Mannelli
Date: 2020-06-25 00:00:00

Science meets Engineering in Deep Learning took place in Vancouver as part of the Workshop section of NeurIPS 2019. As organizers of the workshop, we created the following report in an attempt to isolate emerging topics and recurring themes that have been presented throughout the event. Deep learning can still be a complex mix of art and engineering despite its tremendous success in recent years. The workshop aimed at gathering people across the board to address seemingly contrasting challenges in the problems they are working on. As part of the call for the workshop, particular attention has been given to the interdependence of architecture, data, and optimization that gives rise to an enormous landscape of design and performance intricacies that are not well-understood. This year, our goal was to emphasize the following directions in our community: (i) identify obstacles in the way to better models and algorithms; (ii) identify the general trends from which we would like to build scientific and potentially theoretical understanding; and (iii) the rigorous design of scientific experiments and experimental protocols whose purpose is to resolve and pinpoint the origin of mysteries while ensuring reproducibility and robustness of conclusions. In the event, these topics emerged and were broadly discussed, matching our expectations and paving the way for new studies in these directions. While we acknowledge that the text is naturally biased as it comes through our lens, here we present an attempt to do a fair job of highlighting the outcome of the workshop.
Learning "What-if" Explanations for Sequential Decision-Making
Authors: Ioana Bica, Daniel Jarrett, Alihan Hüyük, Mihaela Van Der Schaar
Date: 2020-07-02 00:00:00
Proceeding: ICLR 2021 1
Tasks: counterfactual, Counterfactual Reasoning, Decision Making, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL), Sequential Decision Making

Building interpretable parameterizations of real-world decision-making on the basis of demonstrated behavior -- i.e. trajectories of observations and actions made by an expert maximizing some unknown reward function -- is essential for introspecting and auditing policies in different institutions. In this paper, we propose learning explanations of expert decisions by modeling their reward function in terms of preferences with respect to "what if" outcomes: Given the current history of observations, what would happen if we took a particular action? To learn these cost-benefit tradeoffs associated with the expert's actions, we integrate counterfactual reasoning into batch inverse reinforcement learning. This offers a principled way of defining reward functions and explaining expert behavior, and also satisfies the constraints of real-world decision-making -- where active experimentation is often impossible (e.g. in healthcare). Additionally, by estimating the effects of different actions, counterfactuals readily tackle the off-policy nature of policy evaluation in the batch setting, and can naturally accommodate settings where the expert policies depend on histories of observations rather than just current states. Through illustrative experiments in both real and simulated medical environments, we highlight the effectiveness of our batch, counterfactual inverse reinforcement learning approach in recovering accurate and interpretable descriptions of behavior.
Discriminative Jackknife: Quantifying Uncertainty in Deep Learning via Higher-Order Influence Functions
Authors: Ahmed M. Alaa, Mihaela van der Schaar
Date: 2020-06-29 00:00:00
Proceeding: ICML 2020 1
Tasks: Uncertainty Quantification

Deep learning models achieve high predictive accuracy across a broad spectrum of tasks, but rigorously quantifying their predictive uncertainty remains challenging. Usable estimates of predictive uncertainty should (1) cover the true prediction targets with high probability, and (2) discriminate between high- and low-confidence prediction instances. Existing methods for uncertainty quantification are based predominantly on Bayesian neural networks; these may fall short of (1) and (2) -- i.e., Bayesian credible intervals do not guarantee frequentist coverage, and approximate posterior inference undermines discriminative accuracy. In this paper, we develop the discriminative jackknife (DJ), a frequentist procedure that utilizes influence functions of a model's loss functional to construct a jackknife (or leave-one-out) estimator of predictive confidence intervals. The DJ satisfies (1) and (2), is applicable to a wide range of deep learning models, is easy to implement, and can be applied in a post-hoc fashion without interfering with model training or compromising its accuracy. Experiments demonstrate that DJ performs competitively compared to existing Bayesian and non-Bayesian regression baselines.
Performance Evaluation of t-SNE and MDS Dimensionality Reduction Techniques with KNN, ENN and SVM Classifiers
Authors: Shadman Sakib, Md. Abu Bakr Siddique, Md. Abdur Rahman
Date: 2020-06-20 00:00:00
Tasks: Classification, Dimensionality Reduction, General Classification

The central goal of this paper is to establish two commonly available dimensionality reduction (DR) methods i.e. t-distributed Stochastic Neighbor Embedding (t-SNE) and Multidimensional Scaling (MDS) in Matlab and to observe their application in several datasets. These DR techniques are applied to nine different datasets namely CNAE9, Segmentation, Seeds, Pima Indians diabetes, Parkinsons, Movement Libras, Mammographic Masses, Knowledge, and Ionosphere acquired from UCI machine learning repository. By applying t-SNE and MDS algorithms, each dataset is transformed to the half of its original dimension by eliminating unnecessary features from the datasets. Subsequently, these datasets with reduced dimensions are fed into three supervised classification algorithms for classification. These classification algorithms are K Nearest Neighbors (KNN), Extended Nearest Neighbors (ENN), and Support Vector Machine (SVM). Again, all these algorithms are implemented in Matlab. The training and test data ratios are maintained as ninety percent: ten percent for each dataset. Upon accuracy observation, the efficiency for every dimensionality technique with availed classification algorithms is analyzed and the performance of each classifier is evaluated.