pwc-archive/papers-with-abstracts huggingface dataset:
Adversarial Robustness for Machine Learning Cyber Defenses Using Log Data
Authors: Kai Steverson, Jonathan Mullin, Metin Ahiskali
Date: 2020-07-29 00:00:00
Tasks: Adversarial Attack, Adversarial Robustness, BIG-bench Machine Learning, Deep Reinforcement Learning

There has been considerable and growing interest in applying machine learning for cyber defenses. One promising approach has been to apply natural language processing techniques to analyze logs data for suspicious behavior. A natural question arises to how robust these systems are to adversarial attacks. Defense against sophisticated attack is of particular concern for cyber defenses. In this paper, we develop a testing framework to evaluate adversarial robustness of machine learning cyber defenses, particularly those focused on log data. Our framework uses techniques from deep reinforcement learning and adversarial natural language processing. We validate our framework using a publicly available dataset and demonstrate that our adversarial attack does succeed against the target systems, revealing a potential vulnerability. We apply our framework to analyze the influence of different levels of dropout regularization and find that higher dropout levels increases robustness. Moreover 90% dropout probability exhibited the highest level of robustness by a significant margin, which suggests unusually high dropout may be necessary to properly protect against adversarial attacks.
Neural Network-based Reconstruction in Compressed Sensing MRI Without Fully-sampled Training Data
Authors: Alan Q. Wang, Adrian V. Dalca, Mert R. Sabuncu
Date: 2020-07-29 00:00:00
Tasks: compressed sensing, Rolling Shutter Correction

Compressed Sensing MRI (CS-MRI) has shown promise in reconstructing under-sampled MR images, offering the potential to reduce scan times. Classical techniques minimize a regularized least-squares cost function using an expensive iterative optimization procedure. Recently, deep learning models have been developed that model the iterative nature of classical techniques by unrolling iterations in a neural network. While exhibiting superior performance, these methods require large quantities of ground-truth images and have shown to be non-robust to unseen data. In this paper, we explore a novel strategy to train an unrolled reconstruction network in an unsupervised fashion by adopting a loss function widely-used in classical optimization schemes. We demonstrate that this strategy achieves lower loss and is computationally cheap compared to classical optimization solvers while also exhibiting superior robustness compared to supervised models. Code is available at https://github.com/alanqrwang/HQSNet.
Mirostat: A Neural Text Decoding Algorithm that Directly Controls Perplexity
Authors: Sourya Basu, Govardana Sachitanandam Ramachandran, Nitish Shirish Keskar, Lav R. Varshney
Date: 2020-07-29 00:00:00
Proceeding: ICLR 2021 1
Tasks: Language Modeling, Language Modelling

Neural text decoding is important for generating high-quality texts using language models. To generate high-quality text, popular decoding algorithms like top-k, top-p (nucleus), and temperature-based sampling truncate or distort the unreliable low probability tail of the language model. Though these methods generate high-quality text after parameter tuning, they are ad hoc. Not much is known about the control they provide over the statistics of the output, which is important since recent reports show text quality is highest for a specific range of likelihoods. Here, first we provide a theoretical analysis of perplexity in top-k, top-p, and temperature sampling, finding that cross-entropy behaves approximately linearly as a function of p in top-p sampling whereas it is a nonlinear function of k in top-k sampling, under Zipfian statistics. We use this analysis to design a feedback-based adaptive top-k text decoding algorithm called mirostat that generates text (of any length) with a predetermined value of perplexity, and thereby high-quality text without any tuning. Experiments show that for low values of k and p in top-k and top-p sampling, perplexity drops significantly with generated text length, which is also correlated with excessive repetitions in the text (the boredom trap). On the other hand, for large values of k and p, we find that perplexity increases with generated text length, which is correlated with incoherence in the text (confusion trap). Mirostat avoids both traps: experiments show that cross-entropy has a near-linear relation with repetition in generated text. This relation is almost independent of the sampling method but slightly dependent on the model used. Hence, for a given language model, control over perplexity also gives control over repetitions. Experiments with human raters for fluency, coherence, and quality further verify our findings.
Advancing Visual Specification of Code Requirements for Graphs
Authors: Dewi Yokelson
Date: 2020-07-29 00:00:00
Tasks: Optical Character Recognition, Optical Character Recognition (OCR)

Researchers in the humanities are among the many who are now exploring the world of big data. They have begun to use programming languages like Python or R and their corresponding libraries to manipulate large data sets and discover brand new insights. One of the major hurdles that still exists is incorporating visualizations of this data into their projects. Visualization libraries can be difficult to learn how to use, even for those with formal training. Yet these visualizations are crucial for recognizing themes and communicating results to not only other researchers, but also the general public. This paper focuses on producing meaningful visualizations of data using machine learning. We allow the user to visually specify their code requirements in order to lower the barrier for humanities researchers to learn how to program visualizations. We use a hybrid model, combining a neural network and optical character recognition to generate the code to create the visualization.
Learning enables adaptation in cooperation for multi-player stochastic games
Authors: Feng Huang, Ming Cao, Long Wang
Date: 2020-07-29 00:00:00

Interactions among individuals in natural populations often occur in a dynamically changing environment. Understanding the role of environmental variation in population dynamics has long been a central topic in theoretical ecology and population biology. However, the key question of how individuals, in the middle of challenging social dilemmas (e.g., the "tragedy of the commons"), modulate their behaviors to adapt to the fluctuation of the environment has not yet been addressed satisfactorily. Utilizing evolutionary game theory and stochastic games, we develop a game-theoretical framework that incorporates the adaptive mechanism of reinforcement learning to investigate whether cooperative behaviors can evolve in the ever-changing group interaction environment. When the action choices of players are just slightly influenced by past reinforcements, we construct an analytical condition to determine whether cooperation can be favored over defection. Intuitively, this condition reveals why and how the environment can mediate cooperative dilemmas. Under our model architecture, we also compare this learning mechanism with two non-learning decision rules, and we find that learning significantly improves the propensity for cooperation in weak social dilemmas, and, in sharp contrast, hinders cooperation in strong social dilemmas. Our results suggest that in complex social-ecological dilemmas, learning enables the adaptation of individuals to varying environments.
Simultaneously Learning Corrections and Error Models for Geometry-based Visual Odometry Methods
Authors: Andrea De Maio, Simon Lacroix
Date: 2020-07-29 00:00:00
Tasks: Autonomous Driving, Visual Odometry

This paper fosters the idea that deep learning methods can be used to complement classical visual odometry pipelines to improve their accuracy and to associate uncertainty models to their estimations. We show that the biases inherent to the visual odometry process can be faithfully learned and compensated for, and that a learning architecture associated with a probabilistic loss function can jointly estimate a full covariance matrix of the residual errors, defining an error model capturing the heteroscedasticity of the process. Experiments on autonomous driving image sequences assess the possibility to concurrently improve visual odometry and estimate an error associated with its outputs.
Learning Video Representations from Textual Web Supervision
Authors: Jonathan C. Stroud, Zhichao Lu, Chen Sun, Jia Deng, Rahul Sukthankar, Cordelia Schmid, David A. Ross
Date: 2020-07-29 00:00:00
Tasks: Action Recognition, Representation Learning

Videos on the Internet are paired with pieces of text, such as titles and descriptions. This text typically describes the most important content in the video, such as the objects in the scene and the actions being performed. Based on this observation, we propose to use text as a method for learning video representations. To accomplish this, we propose a data collection process and use it to collect 70M video clips shared publicly on the Internet, and we then train a model to pair each video with its associated text. We evaluate the model on several down-stream action recognition tasks, including Kinetics, HMDB-51, and UCF-101. We find that this approach is an effective method of pre-training video representations. Specifically, it outperforms all existing methods for self-supervised and cross-modal video representation learning.
#Brexit: Leave or Remain? The Role of User's Community and Diachronic Evolution on Stance Detection
Authors: Mirko Lai, Viviana Patti, Giancarlo Ruffo, Paolo Rosso
Date: 2020-07-29 00:00:00
Tasks: Classification, General Classification, Stance Detection

Interest has grown around the classification of stance that users assume within online debates in recent years. Stance has been usually addressed by considering users posts in isolation, while social studies highlight that social communities may contribute to influence users' opinion. Furthermore, stance should be studied in a diachronic perspective, since it could help to shed light on users' opinion shift dynamics that can be recorded during the debate. We analyzed the political discussion in UK about the BREXIT referendum on Twitter, proposing a novel approach and annotation schema for stance detection, with the main aim of investigating the role of features related to social network community and diachronic stance evolution. Classification experiments show that such features provide very useful clues for detecting stance.
A Development Cycle for Automated Self-Exploration of Robot Behaviors
Authors: Thomas M. Roehr, Daniel Harnack, Hendrik Wöhrle, Felix Wiebe, Moritz Schilling, Oscar Lima, Malte Langosz, Shivesh Kumar, Sirko Straube, Frank Kirchner
Date: 2020-07-29 00:00:00
Tasks: BIG-bench Machine Learning

In this paper we introduce Q-Rock, a development cycle for the automated self-exploration and qualification of robot behaviors. With Q-Rock, we suggest a novel, integrative approach to automate robot development processes. Q-Rock combines several machine learning and reasoning techniques to deal with the increasing complexity in the design of robotic systems. The Q-Rock development cycle consists of three complementary processes: (1) automated exploration of capabilities that a given robotic hardware provides, (2) classification and semantic annotation of these capabilities to generate more complex behaviors, and (3) mapping between application requirements and available behaviors. These processes are based on a graph-based representation of a robot's structure, including hardware and software components. A central, scalable knowledge base enables collaboration of robot designers including mechanical, electrical and systems engineers, software developers and machine learning experts. In this paper we formalize Q-Rock's integrative development cycle and highlight its benefits with a proof-of-concept implementation and a use case demonstration.
Fibonacci and k-Subsecting Recursive Feature Elimination
Authors: Dariusz Brzezinski
Date: 2020-07-29 00:00:00
Tasks: feature selection

Feature selection is a data mining task with the potential of speeding up classification algorithms, enhancing model comprehensibility, and improving learning accuracy. However, finding a subset of features that is optimal in terms of predictive accuracy is usually computationally intractable. Out of several heuristic approaches to dealing with this problem, the Recursive Feature Elimination (RFE) algorithm has received considerable interest from data mining practitioners. In this paper, we propose two novel algorithms inspired by RFE, called Fibonacci- and k-Subsecting Recursive Feature Elimination, which remove features in logarithmic steps, probing the wrapped classifier more densely for the more promising feature subsets. The proposed algorithms are experimentally compared against RFE on 28 highly multidimensional datasets and evaluated in a practical case study involving 3D electron density maps from the Protein Data Bank. The results show that Fibonacci and k-Subsecting Recursive Feature Elimination are capable of selecting a smaller subset of features much faster than standard RFE, while achieving comparable predictive performance.
Force myography benchmark data for hand gesture recognition and transfer learning
Authors: Thomas Buhl Andersen, Rógvi Eliasen, Mikkel Jarlund, Bin Yang
Date: 2020-07-29 00:00:00
Tasks: Gesture Recognition, Hand Gesture Recognition, Hand-Gesture Recognition, Transfer Learning

Force myography has recently gained increasing attention for hand gesture recognition tasks. However, there is a lack of publicly available benchmark data, with most existing studies collecting their own data often with custom hardware and for varying sets of gestures. This limits the ability to compare various algorithms, as well as the possibility for research to be done without first needing to collect data oneself. We contribute to the advancement of this field by making accessible a benchmark dataset collected using a commercially available sensor setup from 20 persons covering 18 unique gestures, in the hope of allowing further comparison of results as well as easier entry into this field of research. We illustrate one use-case for such data, showing how we can improve gesture recognition accuracy by utilising transfer learning to incorporate data from multiple other persons. This also illustrates that the dataset can serve as a benchmark dataset to facilitate research on transfer learning algorithms.
Compressing Deep Neural Networks via Layer Fusion
Authors: James O' Neill, Greg Ver Steeg, Aram Galstyan
Date: 2020-07-29 00:00:00
Tasks: Exponential degradation, Language Modelling, Model Compression

This paper proposes \textit{layer fusion} - a model compression technique that discovers which weights to combine and then fuses weights of similar fully-connected, convolutional and attention layers. Layer fusion can significantly reduce the number of layers of the original network with little additional computation overhead, while maintaining competitive performance. From experiments on CIFAR-10, we find that various deep convolution neural networks can remain within 2\% accuracy points of the original networks up to a compression ratio of 3.33 when iteratively retrained with layer fusion. For experiments on the WikiText-2 language modelling dataset where pretrained transformer models are used, we achieve compression that leads to a network that is 20\% of its original size while being within 5 perplexity points of the original network. We also find that other well-established compression techniques can achieve competitive performance when compared to their original networks given a sufficient number of retraining steps. Generally, we observe a clear inflection point in performance as the amount of compression increases, suggesting a bound on the amount of compression that can be achieved before an exponential degradation in performance.
Dynamic Character Graph via Online Face Clustering for Movie Analysis
Authors: Prakhar Kulshreshtha, Tanaya Guha
Date: 2020-07-29 00:00:00
Tasks: Clustering, Face Clustering, Retrieval

An effective approach to automated movie content analysis involves building a network (graph) of its characters. Existing work usually builds a static character graph to summarize the content using metadata, scripts or manual annotations. We propose an unsupervised approach to building a dynamic character graph that captures the temporal evolution of character interaction. We refer to this as the character interaction graph(CIG). Our approach has two components:(i) an online face clustering algorithm that discovers the characters in the video stream as they appear, and (ii) simultaneous creation of a CIG using the temporal dynamics of the resulting clusters. We demonstrate the usefulness of the CIG for two movie analysis tasks: narrative structure (acts) segmentation, and major character retrieval. Our evaluation on full-length movies containing more than 5000 face tracks shows that the proposed approach achieves superior performance for both the tasks.
Linear Attention Mechanism: An Efficient Attention for Semantic Segmentation
Authors: Rui Li, Jianlin Su, Chenxi Duan, Shunyi Zheng
Date: 2020-07-29 00:00:00
Tasks: Semantic Segmentation

In this paper, to remedy this deficiency, we propose a Linear Attention Mechanism which is approximate to dot-product attention with much less memory and computational costs. The efficient design makes the incorporation between attention mechanisms and neural networks more flexible and versatile. Experiments conducted on semantic segmentation demonstrated the effectiveness of linear attention mechanism. Code is available at https://github.com/lironui/Linear-Attention-Mechanism.
Reliable Tuberculosis Detection using Chest X-ray with Deep Learning, Segmentation and Visualization
Authors: Tawsifur Rahman, Amith Khandakar, Muhammad Abdul Kadir, Khandaker R. Islam, Khandaker F. Islam, Rashid Mazhar, Tahir Hamid, Mohammad T. Islam, Zaid B. Mahbub, Mohamed Arselene Ayari, Muhammad E. H. Chowdhury
Date: 2020-07-29 00:00:00
Tasks: Classification, Data Augmentation, General Classification, Image Segmentation, Semantic Segmentation, Sensitivity, Specificity, Transfer Learning

Tuberculosis (TB) is a chronic lung disease that occurs due to bacterial infection and is one of the top 10 leading causes of death. Accurate and early detection of TB is very important, otherwise, it could be life-threatening. In this work, we have detected TB reliably from the chest X-ray images using image pre-processing, data augmentation, image segmentation, and deep-learning classification techniques. Several public databases were used to create a database of 700 TB infected and 3500 normal chest X-ray images for this study. Nine different deep CNNs (ResNet18, ResNet50, ResNet101, ChexNet, InceptionV3, Vgg19, DenseNet201, SqueezeNet, and MobileNet), which were used for transfer learning from their pre-trained initial weights and trained, validated and tested for classifying TB and non-TB normal cases. Three different experiments were carried out in this work: segmentation of X-ray images using two different U-net models, classification using X-ray images, and segmented lung images. The accuracy, precision, sensitivity, F1-score, specificity in the detection of tuberculosis using X-ray images were 97.07 %, 97.34 %, 97.07 %, 97.14 % and 97.36 % respectively. However, segmented lungs for the classification outperformed than whole X-ray image-based classification and accuracy, precision, sensitivity, F1-score, specificity were 99.9 %, 99.91 %, 99.9 %, 99.9 %, and 99.52 % respectively. The paper also used a visualization technique to confirm that CNN learns dominantly from the segmented lung regions results in higher detection accuracy. The proposed method with state-of-the-art performance can be useful in the computer-aided faster diagnosis of tuberculosis.
Between Subjectivity and Imposition: Power Dynamics in Data Annotation for Computer Vision
Authors: Milagros Miceli, Martin Schuessler, Tianling Yang
Date: 2020-07-29 00:00:00

The interpretation of data is fundamental to machine learning. This paper investigates practices of image data annotation as performed in industrial contexts. We define data annotation as a sense-making practice, where annotators assign meaning to data through the use of labels. Previous human-centered investigations have largely focused on annotators subjectivity as a major cause for biased labels. We propose a wider view on this issue: guided by constructivist grounded theory, we conducted several weeks of fieldwork at two annotation companies. We analyzed which structures, power relations, and naturalized impositions shape the interpretation of data. Our results show that the work of annotators is profoundly informed by the interests, values, and priorities of other actors above their station. Arbitrary classifications are vertically imposed on annotators, and through them, on data. This imposition is largely naturalized. Assigning meaning to data is often presented as a technical matter. This paper shows it is, in fact, an exercise of power with multiple implications for individuals and society.
Performance Analysis of Meta-heuristic Algorithms for a Quadratic Assignment Problem
Authors: Zohreh Raziei, Reza Tavakkoli-Moghaddam, Siavash Tabrizian
Date: 2020-07-29 00:00:00
Tasks: Combinatorial Optimization

A quadratic assignment problem (QAP) is a combinatorial optimization problem that belongs to the class of NP-hard ones. So, it is difficult to solve in the polynomial time even for small instances. Research on the QAP has thus focused on obtaining a method to overcome this problem. Heuristics and meta-heuristics algorithm are prevalent solution methods for this problem. This paper is one of comparative studies to apply different metaheuristic algorithms for solving the QAP. One of the most popular approaches for categorizing meta-heuristic algorithms is based on a search strategy, including (1) local search improvement meta-heuristics and (2) global search-based meta-heuristics. The matter that distinguishes this paper from the other is the comparative performance of local and global search (both EA and SI), in which meta-heuristics that consist of genetic algorithm (GA), particle swarm optimization (PSO), hybrid GA-PSO, grey wolf optimization (GWO), harmony search algorithm (HAS) and simulated annealing (SA). Also, one improvement heuristic algorithm (ie, 2-Opt) is used to compare with others. The PSO, GWO and 2-Opt algorithms are improved to achieve the better comparison toward the other algorithms for evaluation. In order to analysis the comparative advantage of these algorithms, eight different factors are presented. By taking into account all these factors, the test is implemented in six test problems of the QAP Library (QAPLIB) from different sizes. Another contribution of this paper is to measure a strong convergence condition for each algorithm in a new way.