pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning for Posture Control Nonlinear Model System and Noise Identification
Authors: Vittorio Lippi, Thomas Mergner, Christoph Maurer
Date: 2020-06-04 00:00:00

In this work we present a system identification procedure based on Convolutional Neural Networks (CNN) for human posture control models. A usual approach to the study of human posture control consists in the identification of parameters for a control system. In this context, linear models are particularly popular due to the relative simplicity in identifying the required parameters and to analyze the results. Nonlinear models, conversely, are required to predict the real behavior exhibited by human subjects and hence it is desirable to use them in posture control analysis. The use of CNN aims to overcome the heavy computational requirement for the identification of nonlinear models, in order to make the analysis of experimental data less time consuming and, in perspective, to make such analysis feasible in the context of clinical tests. Some potential implications of the method for humanoid robotics are also discussed.
Cracking the Black Box: Distilling Deep Sports Analytics
Authors: Xiangyu Sun, Jack Davis, Oliver Schulte, Guiliang Liu
Date: 2020-06-04 00:00:00
Tasks: Deep Learning, Sports Analytics

This paper addresses the trade-off between Accuracy and Transparency for deep learning applied to sports analytics. Neural nets achieve great predictive accuracy through deep learning, and are popular in sports analytics. But it is hard to interpret a neural net model and harder still to extract actionable insights from the knowledge implicit in it. Therefore, we built a simple and transparent model that mimics the output of the original deep learning model and represents the learned knowledge in an explicit interpretable way. Our mimic model is a linear model tree, which combines a collection of linear models with a regression-tree structure. The tree version of a neural network achieves high fidelity, explains itself, and produces insights for expert stakeholders such as athletes and coaches. We propose and compare several scalable model tree learning heuristics to address the computational challenge from datasets with millions of data points.
Batch greedy maximization of non-submodular functions: Guarantees and applications to experimental design
Authors: Jayanth Jagalur-Mohan, Youssef Marzouk
Date: 2020-06-03 00:00:00
Tasks: Experimental Design

We propose and analyze batch greedy heuristics for cardinality constrained maximization of non-submodular non-decreasing set functions. We consider the standard greedy paradigm, along with its distributed greedy and stochastic greedy variants. Our theoretical guarantees are characterized by the combination of submodularity and supermodularity ratios. We argue how these parameters define tight modular bounds based on incremental gains, and provide a novel reinterpretation of the classical greedy algorithm using the minorize-maximize (MM) principle. Based on that analogy, we propose a new class of methods exploiting any plausible modular bound. In the context of optimal experimental design for linear Bayesian inverse problems, we bound the submodularity and supermodularity ratios when the underlying objective is based on mutual information. We also develop novel modular bounds for the mutual information in this setting, and describe certain connections to polyhedral combinatorics. We discuss how algorithms using these modular bounds relate to established statistical notions such as leverage scores and to more recent efforts such as volume sampling. We demonstrate our theoretical findings on synthetic problems and on a real-world climate monitoring example.
Unveiling Relations in the Industry 4.0 Standards Landscape based on Knowledge Graph Embeddings
Authors: Ariam Rivas, Irlán Grangel-González, Diego Collarana, Jens Lehmann, Maria-Esther Vidal
Date: 2020-06-03 00:00:00
Tasks: Knowledge Graph Embeddings, Knowledge Graphs

Industry~4.0 (I4.0) standards and standardization frameworks have been proposed with the goal of \emph{empowering interoperability} in smart factories. These standards enable the description and interaction of the main components, systems, and processes inside of a smart factory. Due to the growing number of frameworks and standards, there is an increasing need for approaches that automatically analyze the landscape of I4.0 standards. Standardization frameworks classify standards according to their functions into layers and dimensions. However, similar standards can be classified differently across the frameworks, producing, thus, interoperability conflicts among them. Semantic-based approaches that rely on ontologies and knowledge graphs, have been proposed to represent standards, known relations among them, as well as their classification according to existing frameworks. Albeit informative, the structured modeling of the I4.0 landscape only provides the foundations for detecting interoperability issues. Thus, graph-based analytical methods able to exploit knowledge encoded by these approaches, are required to uncover alignments among standards. We study the relatedness among standards and frameworks based on community analysis to discover knowledge that helps to cope with interoperability conflicts between standards. We use knowledge graph embeddings to automatically create these communities exploiting the meaning of the existing relationships. In particular, we focus on the identification of similar standards, i.e., communities of standards, and analyze their properties to detect unknown relations. We empirically evaluate our approach on a knowledge graph of I4.0 standards using the Trans$^*$ family of embedding models for knowledge graph entities. Our results are promising and suggest that relations among standards can be detected accurately.
IterefinE: Iterative KG Refinement Embeddings using Symbolic Knowledge
Authors: Siddhant Arora, Srikanta Bedathur, Maya Ramanath, Deepak Sharma
Date: 2020-06-03 00:00:00
Proceeding: AKBC 2020 6
Tasks: Knowledge Graphs

Knowledge Graphs (KGs) extracted from text sources are often noisy and lead to poor performance in downstream application tasks such as KG-based question answering.While much of the recent activity is focused on addressing the sparsity of KGs by using embeddings for inferring new facts, the issue of cleaning up of noise in KGs through KG refinement task is not as actively studied. Most successful techniques for KG refinement make use of inference rules and reasoning over ontologies. Barring a few exceptions, embeddings do not make use of ontological information, and their performance in KG refinement task is not well understood. In this paper, we present a KG refinement framework called IterefinE which iteratively combines the two techniques - one which uses ontological information and inferences rules, PSL-KGI, and the KG embeddings such as ComplEx and ConvE which do not. As a result, IterefinE is able to exploit not only the ontological information to improve the quality of predictions, but also the power of KG embeddings which (implicitly) perform longer chains of reasoning. The IterefinE framework, operates in a co-training mode and results in explicit type-supervised embedding of the refined KG from PSL-KGI which we call as TypeE-X. Our experiments over a range of KG benchmarks show that the embeddings that we produce are able to reject noisy facts from KG and at the same time infer higher quality new facts resulting in up to 9% improvement of overall weighted F1 score
Community detection in sparse time-evolving graphs with a dynamical Bethe-Hessian
Authors: Lorenzo Dall'Amico, Romain Couillet, Nicolas Tremblay
Date: 2020-06-03 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Community Detection, Stochastic Block Model

This article considers the problem of community detection in sparse dynamical graphs in which the community structure evolves over time. A fast spectral algorithm based on an extension of the Bethe-Hessian matrix is proposed, which benefits from the positive correlation in the class labels and in their temporal evolution and is designed to be applicable to any dynamical graph with a community structure. Under the dynamical degree-corrected stochastic block model, in the case of two classes of equal size, we demonstrate and support with extensive simulations that our proposed algorithm is capable of making non-trivial community reconstruction as soon as theoretically possible, thereby reaching the optimal detectability threshold and provably outperforming competing spectral methods.
Classifying histograms of medical data using information geometry of beta distributions
Authors: Alice Le Brigant, Nicolas Guigui, Sana Rebbah, Stéphane Puechmorel
Date: 2020-06-03 00:00:00

In this paper, we use tools of information geometry to compare, average and classify histograms. Beta distributions are fitted to the histograms and the corresponding Fisher information geometry is used for comparison. We show that this geometry is negatively curved, which guarantees uniqueness of the notion of mean, and makes it suitable to classify histograms through the popular K-means algorithm. We illustrate the use of these geometric tools in supervised and unsupervised classification procedures of two medical data-sets, cardiac shape deformations for the detection of pulmonary hypertension and brain cortical thickness for the diagnosis of Alzheimer's disease.
Objective-Sensitive Principal Component Analysis for High-Dimensional Inverse Problems
Authors: Maksim Elizarev, Andrei Mukhin, Aleksey Khlyupin
Date: 2020-06-02 00:00:00
Tasks: Vocal Bursts Intensity Prediction

We present a novel approach for adaptive, differentiable parameterization of large-scale random fields. If the approach is coupled with any gradient-based optimization algorithm, it can be applied to a variety of optimization problems, including history matching. The developed technique is based on principal component analysis (PCA) but modifies a purely data-driven basis of principal components considering objective function behavior. To define an efficient encoding, Gradient-Sensitive PCA uses an objective function gradient with respect to model parameters. We propose computationally efficient implementations of the technique, and two of them are based on stationary perturbation theory (SPT). Optimality, correctness, and low computational costs of the new encoding approach are tested, verified, and discussed. Three algorithms for optimal parameter decomposition are presented and applied to an objective of 2D synthetic history matching. The results demonstrate improvements in encoding quality regarding objective function minimization and distributional patterns of the desired field. Possible applications and extensions are proposed.
Maximizing Cumulative User Engagement in Sequential Recommendation: An Online Optimization Perspective
Authors: Yifei Zhao, Yu-Hang Zhou, Mingdong Ou, Huan Xu, Nan Li
Date: 2020-06-02 00:00:00
Tasks: Sequential Recommendation

To maximize cumulative user engagement (e.g. cumulative clicks) in sequential recommendation, it is often needed to tradeoff two potentially conflicting objectives, that is, pursuing higher immediate user engagement (e.g., click-through rate) and encouraging user browsing (i.e., more items exposured). Existing works often study these two tasks separately, thus tend to result in sub-optimal results. In this paper, we study this problem from an online optimization perspective, and propose a flexible and practical framework to explicitly tradeoff longer user browsing length and high immediate user engagement. Specifically, by considering items as actions, user's requests as states and user leaving as an absorbing state, we formulate each user's behavior as a personalized Markov decision process (MDP), and the problem of maximizing cumulative user engagement is reduced to a stochastic shortest path (SSP) problem. Meanwhile, with immediate user engagement and quit probability estimation, it is shown that the SSP problem can be efficiently solved via dynamic programming. Experiments on real-world datasets demonstrate the effectiveness of the proposed approach. Moreover, this approach is deployed at a large E-commerce platform, achieved over 7% improvement of cumulative clicks.
Dimensionality Reduction for Sentiment Classification: Evolving for the Most Prominent and Separable Features
Authors: Aftab Anjum, Mazharul Islam, Lin Wang
Date: 2020-06-01 00:00:00
Tasks: Dimensionality Reduction, General Classification, Sentiment Analysis, Sentiment Classification

In sentiment classification, the enormous amount of textual data, its immense dimensionality, and inherent noise make it extremely difficult for machine learning classifiers to extract high-level and complex abstractions. In order to make the data less sparse and more statistically significant, the dimensionality reduction techniques are needed. But in the existing dimensionality reduction techniques, the number of components needs to be set manually which results in loss of the most prominent features, thus reducing the performance of the classifiers. Our prior work, i.e., Term Presence Count (TPC) and Term Presence Ratio (TPR) have proven to be effective techniques as they reject the less separable features. However, the most prominent and separable features might still get removed from the initial feature set despite having higher distributions among positive and negative tagged documents. To overcome this problem, we have proposed a new framework that consists of two-dimensionality reduction techniques i.e., Sentiment Term Presence Count (SentiTPC) and Sentiment Term Presence Ratio (SentiTPR). These techniques reject the features by considering term presence difference for SentiTPC and ratio of the distribution distinction for SentiTPR. Additionally, these methods also analyze the total distribution information. Extensive experimental results exhibit that the proposed framework reduces the feature dimension by a large scale, and thus significantly improve the classification performance.
Jointly Modeling Intra- and Inter-transaction Dependencies with Hierarchical Attentive Transaction Embeddings for Next-item Recommendation
Authors: Shoujin Wang, Longbing Cao, Liang Hu, Shlomo Berkovsky, Xiaoshui Huang, Lin Xiao, Wenpeng Lu
Date: 2020-05-30 00:00:00
Tasks: Recommendation Systems

A transaction-based recommender system (TBRS) aims to predict the next item by modeling dependencies in transactional data. Generally, two kinds of dependencies considered are intra-transaction dependency and inter-transaction dependency. Most existing TBRSs recommend next item by only modeling the intra-transaction dependency within the current transaction while ignoring inter-transaction dependency with recent transactions that may also affect the next item. However, as not all recent transactions are relevant to the current and next items, the relevant ones should be identified and prioritized. In this paper, we propose a novel hierarchical attentive transaction embedding (HATE) model to tackle these issues. Specifically, a two-level attention mechanism integrates both item embedding and transaction embedding to build an attentive context representation that incorporates both intraand inter-transaction dependencies. With the learned context representation, HATE then recommends the next item. Experimental evaluations on two real-world transaction datasets show that HATE significantly outperforms the state-ofthe-art methods in terms of recommendation accuracy.
Detecting Problem Statements in Peer Assessments
Authors: Yunkai Xiao, Gabriel Zingle, Qinjin Jia, Harsh R. Shah, Yi Zhang, Tianyi Li, Mohsin Karovaliya, Weixiang Zhao, Yang song, Jie Ji, Ashwin Balasubramaniam, Harshit Patel, Priyankha Bhalasubbramanian, Vikram Patel, Edward F. Gehringer
Date: 2020-05-30 00:00:00

Effective peer assessment requires students to be attentive to the deficiencies in the work they rate. Thus, their reviews should identify problems. But what ways are there to check that they do? We attempt to automate the process of deciding whether a review comment detects a problem. We use over 18,000 review comments that were labeled by the reviewees as either detecting or not detecting a problem with the work. We deploy several traditional machine-learning models, as well as neural-network models using GloVe and BERT embeddings. We find that the best performer is the Hierarchical Attention Network classifier, followed by the Bidirectional Gated Recurrent Units (GRU) Attention and Capsule model with scores of 93.1% and 90.5% respectively. The best non-neural network model was the support vector machine with a score of 89.71%. This is followed by the Stochastic Gradient Descent model and the Logistic Regression model with 89.70% and 88.98%.
Adversarial Robustness of Deep Convolutional Candlestick Learner
Authors: Jun-Hao Chen, Samuel Yen-Chi Chen, Yun-Cheng Tsai, Chih-Shiang Shur
Date: 2020-05-29 00:00:00
Tasks: Adversarial Robustness

Deep learning (DL) has been applied extensively in a wide range of fields. However, it has been shown that DL models are susceptible to a certain kinds of perturbations called \emph{adversarial attacks}. To fully unlock the power of DL in critical fields such as financial trading, it is necessary to address such issues. In this paper, we present a method of constructing perturbed examples and use these examples to boost the robustness of the model. Our algorithm increases the stability of DL models for candlestick classification with respect to perturbations in the input data.
Investigation Into the Viability of Neural Networks as a Means for Anomaly Detection in Experiments Like Atlas at the LHC
Authors: Sully Billingsley
Date: 2020-05-29 00:00:00
Tasks: Anomaly Detection

Petabytes of data are generated at the Atlas experiment at the Large Hadron Collider however not all of it is necessarily interesting, so what do we do with all of this data and how do we find these interesting needles in an uninteresting haystack. This problem can possibly be solved through the process of anomaly detection. In this document, Investigation Into the Viability of Neural Networks as a Means for Anomaly Detection in Experiments Like Atlas at the LHC the effectiveness of different types of neural network architectures as anomaly detectors are researched using Monte Carlo simulated data generated by the DarkMachines project. This data is meant to replicate Standard Model and Beyond Standard Model events. By finding an effective model, the Atlas experiment can become more effective and fewer interesting events will be lost.
BERT-XML: Large Scale Automated ICD Coding Using BERT Pretraining
Authors: Zachariah Zhang, Jingshu Liu, Narges Razavian
Date: 2020-05-26 00:00:00
Proceeding: EMNLP (ClinicalNLP) 2020 11

Clinical interactions are initially recorded and documented in free text medical notes. ICD coding is the task of classifying and coding all diagnoses, symptoms and procedures associated with a patient's visit. The process is often manual and extremely time-consuming and expensive for hospitals. In this paper, we propose a machine learning model, BERT-XML, for large scale automated ICD coding from EHR notes, utilizing recently developed unsupervised pretraining that have achieved state of the art performance on a variety of NLP tasks. We train a BERT model from scratch on EHR notes, learning with vocabulary better suited for EHR tasks and thus outperform off-the-shelf models. We adapt the BERT architecture for ICD coding with multi-label attention. While other works focus on small public medical datasets, we have produced the first large scale ICD-10 classification model using millions of EHR notes to predict thousands of unique ICD codes.
AutoMSC: Automatic Assignment of Mathematics Subject Classification Labels
Authors: Moritz Schubotz, Philipp Scharpf, Olaf Teschke, Andreas Kuehnemund, Corinna Breitinger, Bela Gipp
Date: 2020-05-25 00:00:00
Tasks: Articles, Classification, General Classification, Math, Multi-class Classification

Authors of research papers in the fields of mathematics, and other math-heavy disciplines commonly employ the Mathematics Subject Classification (MSC) scheme to search for relevant literature. The MSC is a hierarchical alphanumerical classification scheme that allows librarians to specify one or multiple codes for publications. Digital Libraries in Mathematics, as well as reviewing services, such as zbMATH and Mathematical Reviews (MR) rely on these MSC labels in their workflows to organize the abstracting and reviewing process. Especially, the coarse-grained classification determines the subject editor who is responsible for the actual reviewing process. In this paper, we investigate the feasibility of automatically assigning a coarse-grained primary classification using the MSC scheme, by regarding the problem as a multi-class classification machine learning task. We find that our method achieves an (F_1)-score of over 77%, which is remarkably close to the agreement of zbMATH and MR ((F_1)-score of 81%). Moreover, we find that the method's confidence score allows for reducing the effort by 86% compared to the manual coarse-grained classification effort while maintaining a precision of 81% for automatically classified articles.
Host-Pathongen Co-evolution Inspired Algorithm Enables Robust GAN Training
Authors: Andrei Kucharavy, El Mahdi El Mhamdi, Rachid Guerraoui
Date: 2020-05-22 00:00:00

Generative adversarial networks (GANs) are pairs of artificial neural networks that are trained one against each other. The outputs from a generator are mixed with the real-world inputs to the discriminator and both networks are trained until an equilibrium is reached, where the discriminator cannot distinguish generated inputs from real ones. Since their introduction, GANs have allowed for the generation of impressive imitations of real-life films, images and texts, whose fakeness is barely noticeable to humans. Despite their impressive performance, training GANs remains to this day more of an art than a reliable procedure, in a large part due to training process stability. Generators are susceptible to mode dropping and convergence to random patterns, which have to be mitigated by computationally expensive multiple restarts. Curiously, GANs bear an uncanny similarity to a co-evolution of a pathogen and its host's immune system in biology. In a biological context, the majority of potential pathogens indeed never make it and are kept at bay by the hots' immune system. Yet some are efficient enough to present a risk of a serious condition and recurrent infections. Here, we explore that similarity to propose a more robust algorithm for GANs training. We empirically show the increased stability and a better ability to generate high-quality images while using less computational power.