pwc-archive/papers-with-abstracts huggingface dataset:
Recurrent Neural Network Learning of Performance and Intrinsic Population Dynamics from Sparse Neural Data
Authors: Alessandro Salatiello, Martin A. Giese
Date: 2020-05-05 00:00:00

Recurrent Neural Networks (RNNs) are popular models of brain function. The typical training strategy is to adjust their input-output behavior so that it matches that of the biological circuit of interest. Even though this strategy ensures that the biological and artificial networks perform the same computational task, it does not guarantee that their internal activity dynamics match. This suggests that the trained RNNs might end up performing the task employing a different internal computational mechanism, which would make them a suboptimal model of the biological circuit. In this work, we introduce a novel training strategy that allows learning not only the input-output behavior of an RNN but also its internal network dynamics, based on sparse neural recordings. We test the proposed method by training an RNN to simultaneously reproduce internal dynamics and output signals of a physiologically-inspired neural model. Specifically, this model generates the multiphasic muscle-like activity patterns typically observed during the execution of reaching movements, based on the oscillatory activation patterns concurrently observed in the motor cortex. Remarkably, we show that the reproduction of the internal dynamics is successful even when the training algorithm relies on the activities of a small subset of neurons sampled from the biological network. Furthermore, we show that training the RNNs with this method significantly improves their generalization performance. Overall, our results suggest that the proposed method is suitable for building powerful functional RNN models, which automatically capture important computational properties of the biological circuit of interest from sparse neural recordings.
When Machine Unlearning Jeopardizes Privacy
Authors: Min Chen, Zhikun Zhang, Tianhao Wang, Michael Backes, Mathias Humbert, Yang Zhang
Date: 2020-05-05 00:00:00
Tasks: Inference Attack, Machine Unlearning, Membership Inference Attack

The right to be forgotten states that a data owner has the right to erase their data from an entity storing it. In the context of machine learning (ML), the right to be forgotten requires an ML model owner to remove the data owner's data from the training set used to build the ML model, a process known as machine unlearning. While originally designed to protect the privacy of the data owner, we argue that machine unlearning may leave some imprint of the data in the ML model and thus create unintended privacy risks. In this paper, we perform the first study on investigating the unintended information leakage caused by machine unlearning. We propose a novel membership inference attack that leverages the different outputs of an ML model's two versions to infer whether a target sample is part of the training set of the original model but out of the training set of the corresponding unlearned model. Our experiments demonstrate that the proposed membership inference attack achieves strong performance. More importantly, we show that our attack in multiple cases outperforms the classical membership inference attack on the original ML model, which indicates that machine unlearning can have counterproductive effects on privacy. We notice that the privacy degradation is especially significant for well-generalized ML models where classical membership inference does not perform well. We further investigate four mechanisms to mitigate the newly discovered privacy risks and show that releasing the predicted label only, temperature scaling, and differential privacy are effective. We believe that our results can help improve privacy protection in practical implementations of machine unlearning. Our code is available at https://github.com/MinChen00/UnlearningLeaks.
Measuring the Discrepancy between Conditional Distributions: Methods, Properties and Applications
Authors: Shujian Yu, Ammar Shaker, Francesco Alesiani, Jose C. Principe
Date: 2020-05-05 00:00:00
Tasks: Drift Detection, feature selection, Multi-Task Learning

We propose a simple yet powerful test statistic to quantify the discrepancy between two conditional distributions. The new statistic avoids the explicit estimation of the underlying distributions in highdimensional space and it operates on the cone of symmetric positive semidefinite (SPS) matrix using the Bregman matrix divergence. Moreover, it inherits the merits of the correntropy function to explicitly incorporate high-order statistics in the data. We present the properties of our new statistic and illustrate its connections to prior art. We finally show the applications of our new statistic on three different machine learning problems, namely the multi-task learning over graphs, the concept drift detection, and the information-theoretic feature selection, to demonstrate its utility and advantage. Code of our statistic is available at https://bit.ly/BregmanCorrentropy.
Reproduction of Lateral Inhibition-Inspired Convolutional Neural Network for Visual Attention and Saliency Detection
Authors: Filip Marcinek
Date: 2020-05-05 00:00:00
Tasks: Object, Saliency Detection

In recent years, neural networks have continued to flourish, achieving high efficiency in detecting relevant objects in photos or simply recognizing (classifying) these objects - mainly using CNN networks. Current solutions, however, are far from ideal, because it often turns out that network can be effectively confused with even natural images examples. I suspect that the classification of an object is strongly influenced by the background pixels on which the object is located. In my work, I analyze the above problem using for this purpose saliency maps created by the LICNN network. They are designed to suppress the neurons surrounding the examined object and, consequently, reduce the contribution of background pixels to the classifier predictions. My experiments on the natural and adversarial images datasets show that, indeed, there is a visible correlation between the background and the wrong-classified foreground object. This behavior of the network is not supported by human experience, because, for example, we do not confuse the yellow school bus with the snow plow just because it is on the snowy background.
II-20: Intelligent and pragmatic analytic categorization of image collections
Authors: Jan Zahálka, Marcel Worring, Jarke J. van Wijk
Date: 2020-05-05 00:00:00

We introduce II-20 (Image Insight 2020), a multimedia analytics approach for analytic categorization of image collections. Advanced visualizations for image collections exist, but they need tight integration with a machine model to support analytic categorization. Directly employing computer vision and interactive learning techniques gravitates towards search. Analytic categorization, however, is not machine classification (the difference between the two is called the pragmatic gap): a human adds/redefines/deletes categories of relevance on the fly to build insight, whereas the machine classifier is rigid and non-adaptive. Analytic categorization that brings the user to insight requires a flexible machine model that allows dynamic sliding on the exploration-search axis, as well as semantic interactions. II-20 brings 3 major contributions to multimedia analytics on image collections and towards closing the pragmatic gap. Firstly, a machine model that closely follows the user's interactions and dynamically models her categories of relevance. II-20's model, in addition to matching and exceeding the state of the art w. r. t. relevance, allows the user to dynamically slide on the exploration-search axis without additional input from her side. Secondly, the dynamic, 1-image-at-a-time Tetris metaphor that synergizes with the model. It allows the model to analyze the collection by itself with minimal interaction from the user and complements the classic grid metaphor. Thirdly, the fast-forward interaction, allowing the user to harness the model to quickly expand ("fast-forward") the categories of relevance, expands the multimedia analytics semantic interaction dictionary. Automated experiments show that II-20's model outperforms the state of the art and also demonstrate Tetris's analytic quality. User studies confirm that II-20 is an intuitive, efficient, and effective multimedia analytics tool.
Artemis: A Novel Annotation Methodology for Indicative Single Document Summarization
Authors: Rahul Jha, Keping Bi, Yang Li, Mahdi Pakdaman, Asli Celikyilmaz, Ivan Zhiboedov, Kieran McDonald
Date: 2020-05-05 00:00:00
Proceeding: EMNLP (Eval4NLP) 2020 11
Tasks: Articles, Document Summarization, Information Retrieval, Management, Retrieval, Sentence

We describe Artemis (Annotation methodology for Rich, Tractable, Extractive, Multi-domain, Indicative Summarization), a novel hierarchical annotation process that produces indicative summaries for documents from multiple domains. Current summarization evaluation datasets are single-domain and focused on a few domains for which naturally occurring summaries can be easily found, such as news and scientific articles. These are not sufficient for training and evaluation of summarization models for use in document management and information retrieval systems, which need to deal with documents from multiple domains. Compared to other annotation methods such as Relative Utility and Pyramid, Artemis is more tractable because judges don't need to look at all the sentences in a document when making an importance judgment for one of the sentences, while providing similarly rich sentence importance annotations. We describe the annotation process in detail and compare it with other similar evaluation systems. We also present analysis and experimental results over a sample set of 532 annotated documents.
Stealing Links from Graph Neural Networks
Authors: Xinlei He, Jinyuan Jia, Michael Backes, Neil Zhenqiang Gong, Yang Zhang
Date: 2020-05-05 00:00:00
Tasks: Fraud Detection, Recommendation Systems

Graph data, such as chemical networks and social networks, may be deemed confidential/private because the data owner often spends lots of resources collecting the data or the data contains sensitive information, e.g., social relationships. Recently, neural networks were extended to graph data, which are known as graph neural networks (GNNs). Due to their superior performance, GNNs have many applications, such as healthcare analytics, recommender systems, and fraud detection. In this work, we propose the first attacks to steal a graph from the outputs of a GNN model that is trained on the graph. Specifically, given a black-box access to a GNN model, our attacks can infer whether there exists a link between any pair of nodes in the graph used to train the model. We call our attacks link stealing attacks. We propose a threat model to systematically characterize an adversary's background knowledge along three dimensions which in total leads to a comprehensive taxonomy of 8 different link stealing attacks. We propose multiple novel methods to realize these 8 attacks. Extensive experiments on 8 real-world datasets show that our attacks are effective at stealing links, e.g., AUC (area under the ROC curve) is above 0.95 in multiple cases. Our results indicate that the outputs of a GNN model reveal rich information about the structure of the graph used to train the model.
Adaptive Interaction Modeling via Graph Operations Search
Authors: Haoxin Li, Wei-Shi Zheng, Yu Tao, Haifeng Hu, Jian-Huang Lai
Date: 2020-05-05 00:00:00
Proceeding: CVPR 2020 6
Tasks: Action Analysis

Interaction modeling is important for video action analysis. Recently, several works design specific structures to model interactions in videos. However, their structures are manually designed and non-adaptive, which require structures design efforts and more importantly could not model interactions adaptively. In this paper, we automate the process of structures design to learn adaptive structures for interaction modeling. We propose to search the network structures with differentiable architecture search mechanism, which learns to construct adaptive structures for different videos to facilitate adaptive interaction modeling. To this end, we first design the search space with several basic graph operations that explicitly capture different relations in videos. We experimentally demonstrate that our architecture search framework learns to construct adaptive interaction modeling structures, which provides more understanding about the relations between the structures and some interaction characteristics, and also releases the requirement of structures design efforts. Additionally, we show that the designed basic graph operations in the search space are able to model different interactions in videos. The experiments on two interaction datasets show that our method achieves competitive performance with state-of-the-arts.
Effect of the sEMG electrode (re)placement and feature set size on the hand movement recognition
Authors: Nadica Miljković, Milica S. Isaković
Date: 2020-05-05 00:00:00
Tasks: Classification, General Classification

Repositioning of recording electrode array across repeated electromyography measurements may result in a displacement error in hand movement classification systems. In order to examine if the classifier re-training could reach satisfactory results when electrode array is translated along or rotated around subject's forearm for varying number of features, we recorded surface electromyography signals in 10 healthy volunteers for three types of grasp and 6 wrist movements. For feature extraction we applied principal component analysis and the feature set size varied from one to 8 principal components. We compared results of re-trained classifier with results from leave-one-out cross-validation classification procedure for three classifiers: LDA (Linear Discriminant Analysis), QDA (Quadratic Discriminant Analysis), and ANN (Artificial Neural Network). Our results showed that there was no significant difference in classification accuracy when the array electrode was repositioned indicating successful classification re-training and optimal feature set selection. The results also indicate expectedly that the number of principal components plays a key role for acceptable classification accuracy ~90 %. For the largest dataset (9 hand movements), LDA and QDA outperformed ANN, while for three grasping movements ANN showed promising results. Interestingly, we showed that interaction between electrode array position and the feature set size is not statistically significant. This study emphasizes the importance of testing the interaction of factors that influence classification accuracy and classifier selection altogether with their impact independently in order to establish guiding principles for design of hand movement recognition system. Data recorded for this study are stored on Zenodo repository (doi: 10.5281/zenodo.4039550).
Probing Criticality in Quantum Spin Chains with Neural Networks
Authors: A Berezutskii, M Beketov, D Yudin, Z Zimborás, J. Biamonte
Date: 2020-05-05 00:00:00
Tasks: Dimensionality Reduction

The numerical emulation of quantum systems often requires an exponential number of degrees of freedom which translates to a computational bottleneck. Methods of machine learning have been used in adjacent fields for effective feature extraction and dimensionality reduction of high-dimensional datasets. Recent studies have revealed that neural networks are further suitable for the determination of macroscopic phases of matter and associated phase transitions as well as efficient quantum state representation. In this work, we address quantum phase transitions in quantum spin chains, namely the transverse field Ising chain and the anisotropic XY chain, and show that even neural networks with no hidden layers can be effectively trained to distinguish between magnetically ordered and disordered phases. Our neural network acts to predict the corresponding crossovers finite-size systems undergo. Our results extend to a wide class of interacting quantum many-body systems and illustrate the wide applicability of neural networks to many-body quantum physics.
Superposition for Lambda-Free Higher-Order Logic
Authors: Alexander Bentkamp, Jasmin Blanchette, Simon Cruanes, Uwe Waldmann
Date: 2020-05-05 00:00:00

We introduce refutationally complete superposition calculi for intentional and extensional clausal $\lambda$-free higher-order logic, two formalisms that allow partial application and applied variables. The calculi are parameterized by a term order that need not be fully monotonic, making it possible to employ the $\lambda$-free higher-order lexicographic path and Knuth-Bendix orders. We implemented the calculi in the Zipperposition prover and evaluated them on Isabelle/HOL and TPTP benchmarks. They appear promising as a stepping stone towards complete, highly efficient automatic theorem provers for full higher-order logic.
Self-organizing Pattern in Multilayer Network for Words and Syllables
Authors: Li-Min Wang, Sun-Ting Tsai, Shan-Jyun Wu, Meng-Xue Tsai, Daw-Wei Wang, Yi-Ching Su, Tzay-Ming Hong
Date: 2020-05-05 00:00:00

One of the ultimate goals for linguists is to find universal properties in human languages. Although words are generally considered as representing arbitrary mapping between linguistic forms and meanings, we propose a new universal law that highlights the equally important role of syllables, which is complementary to Zipf's. By plotting rank-rank frequency distribution of word and syllable for English and Chinese corpora, visible lines appear and can be fit to a master curve. We discover the multi-layer network for words and syllables based on this analysis exhibits the feature of self-organization which relies heavily on the inclusion of syllables and their connections. Analytic form for the scaling structure is derived and used to quantify how Internet slang becomes fashionable, which demonstrates its usefulness as a new tool to evolutionary linguistics.
Using Machine Learning to Emulate Agent-Based Simulations
Authors: Claudio Angione, Eric Silverman, Elisabeth Yaneske
Date: 2020-05-05 00:00:00
Tasks: BIG-bench Machine Learning, CPU

In this proof-of-concept work, we evaluate the performance of multiple machine-learning methods as statistical emulators for use in the analysis of agent-based models (ABMs). Analysing ABM outputs can be challenging, as the relationships between input parameters can be non-linear or even chaotic even in relatively simple models, and each model run can require significant CPU time. Statistical emulation, in which a statistical model of the ABM is constructed to facilitate detailed model analyses, has been proposed as an alternative to computationally costly Monte Carlo methods. Here we compare multiple machine-learning methods for ABM emulation in order to determine the approaches best suited to emulating the complex behaviour of ABMs. Our results suggest that, in most scenarios, artificial neural networks (ANNs) and gradient-boosted trees outperform Gaussian process emulators, currently the most commonly used method for the emulation of complex computational models. ANNs produced the most accurate model replications in scenarios with high numbers of model runs, although training times were longer than the other methods. We propose that agent-based modelling would benefit from using machine-learning methods for emulation, as this can facilitate more robust sensitivity analyses for the models while also reducing CPU time consumption when calibrating and analysing the simulation.
Explainable AI for Classification using Probabilistic Logic Inference
Authors: Xiuyi Fan, Siyuan Liu, Thomas C. Henderson
Date: 2020-05-05 00:00:00
Tasks: Classification, General Classification

The overarching goal of Explainable AI is to develop systems that not only exhibit intelligent behaviours, but also are able to explain their rationale and reveal insights. In explainable machine learning, methods that produce a high level of prediction accuracy as well as transparent explanations are valuable. In this work, we present an explainable classification method. Our method works by first constructing a symbolic Knowledge Base from the training data, and then performing probabilistic inferences on such Knowledge Base with linear programming. Our approach achieves a level of learning performance comparable to that of traditional classifiers such as random forests, support vector machines and neural networks. It identifies decisive features that are responsible for a classification as explanations and produces results similar to the ones found by SHAP, a state of the art Shapley Value based method. Our algorithms perform well on a range of synthetic and non-synthetic data sets.
Encoding Linear Constraints into SAT
Authors: Ignasi Abío, Valentin Mayer-Eichberger, Peter Stuckey
Date: 2020-05-05 00:00:00

Linear integer constraints are one of the most important constraints in combinatorial problems since they are commonly found in many practical applications. Typically, encodings to Boolean satisfiability (SAT) format of conjunctive normal form perform poorly in problems with these constraints in comparison with SAT modulo theories (SMT), lazy clause generation (LCG) or mixed integer programming (MIP) solvers. In this paper we explore and categorize SAT encodings for linear integer constraints. We define new SAT encodings based on multi-valued decision diagrams, and sorting networks. We compare different SAT encodings of linear constraints and demonstrate where one may be preferable to another. We also compare SAT encodings against other solving methods and show they can be better than linear integer (MIP) solvers and sometimes better than LCG or SMT solvers on appropriate problems. Combining the new encoding with lazy decomposition, which during runtime only encodes constraints that are important to the solving process that occurs, gives the best option for many highly combinatorial problems involving linear constraints.
Automatic Tracking of the Muscle Tendon Junction in Healthy and Impaired Subjects using Deep Learning
Authors: Christoph Leitner, Robert Jarolim, Andreas Konrad, Annika Kruse, Markus Tilp, Jörg Schröttner, Christian Baumgartner
Date: 2020-05-05 00:00:00
Tasks: Muscle Tendon Junction Identification

Recording muscle tendon junction displacements during movement, allows separate investigation of the muscle and tendon behaviour, respectively. In order to provide a fully-automatic tracking method, we employ a novel deep learning approach to detect the position of the muscle tendon junction in ultrasound images. We utilize the attention mechanism to enable the network to focus on relevant regions and to obtain a better interpretation of the results. Our data set consists of a large cohort of 79 healthy subjects and 28 subjects with movement limitations performing passive full range of motion and maximum contraction movements. Our trained network shows robust detection of the muscle tendon junction on a diverse data set of varying quality with a mean absolute error of 2.55$\pm$1 mm. We show that our approach can be applied for various subjects and can be operated in real-time. The complete software package is available for open-source use via: https://github.com/luuleitner/deepMTJ