pwc-archive/papers-with-abstracts huggingface dataset:
GAMA: a General Automated Machine learning Assistant
Authors: Pieter Gijsbers, Joaquin Vanschoren
Date: 2020-07-09 00:00:00
Tasks: AutoML, Benchmarking, BIG-bench Machine Learning

The General Automated Machine learning Assistant (GAMA) is a modular AutoML system developed to empower users to track and control how AutoML algorithms search for optimal machine learning pipelines, and facilitate AutoML research itself. In contrast to current, often black-box systems, GAMA allows users to plug in different AutoML and post-processing techniques, logs and visualizes the search process, and supports easy benchmarking. It currently features three AutoML search algorithms, two model post-processing steps, and is designed to allow for more components to be added.
Modified Possibilistic Fuzzy C-Means Algorithm for Clustering Incomplete Data Sets
Authors: Rustam, Koredianto Usman, Mudyawati Kamaruddin, Dina Chamidah, Nopendri, Khaerudin Saleh, Yulinda Eliskar, Ismail Marzuki
Date: 2020-07-09 00:00:00
Tasks: Clustering

Possibilistic fuzzy c-means (PFCM) algorithm is a reliable algorithm has been proposed to deal the weakness of two popular algorithms for clustering, fuzzy c-means (FCM) and possibilistic c-means (PCM). PFCM algorithm deals with the weaknesses of FCM in handling noise sensitivity and the weaknesses of PCM in the case of coincidence clusters. However, the PFCM algorithm can be only applied to cluster complete data sets. Therefore, in this study, we propose a modification of the PFCM algorithm that can be applied to incomplete data sets clustering. We modified the PFCM algorithm to OCSPFCM and NPSPFCM algorithms and measured performance on three things: 1) accuracy percentage, 2) a number of iterations to termination, and 3) centroid errors. Based on the results that both algorithms have the potential for clustering incomplete data sets. However, the performance of the NPSPFCM algorithm is better than the OCSPFCM algorithm for clustering incomplete data sets.
Uncertainty Quantification in Deep Residual Neural Networks
Authors: Lukasz Wandzik, Raul Vicente Garcia, Jörg Krüger
Date: 2020-07-09 00:00:00
Tasks: Uncertainty Quantification

Uncertainty quantification is an important and challenging problem in deep learning. Previous methods rely on dropout layers which are not present in modern deep architectures or batch normalization which is sensitive to batch sizes. In this work, we address the problem of uncertainty quantification in deep residual networks by using a regularization technique called stochastic depth. We show that training residual networks using stochastic depth can be interpreted as a variational approximation to the intractable posterior over the weights in Bayesian neural networks. We demonstrate that by sampling from a distribution of residual networks with varying depth and shared weights, meaningful uncertainty estimates can be obtained. Moreover, compared to the original formulation of residual networks, our method produces well-calibrated softmax probabilities with only minor changes to the network's structure. We evaluate our approach on popular computer vision datasets and measure the quality of uncertainty estimates. We also test the robustness to domain shift and show that our method is able to express higher predictive uncertainty on out-of-distribution samples. Finally, we demonstrate how the proposed approach could be used to obtain uncertainty estimates in facial verification applications.
Cross-Modal Weighting Network for RGB-D Salient Object Detection
Authors: Gongyang Li, Zhi Liu, Linwei Ye, Yang Wang, Haibin Ling
Date: 2020-07-09 00:00:00
Proceeding: ECCV 2020 8
Tasks: object-detection, Object Detection, Object Localization, RGB-D Salient Object Detection, RGB Salient Object Detection, Salient Object Detection

Depth maps contain geometric clues for assisting Salient Object Detection (SOD). In this paper, we propose a novel Cross-Modal Weighting (CMW) strategy to encourage comprehensive interactions between RGB and depth channels for RGB-D SOD. Specifically, three RGB-depth interaction modules, named CMW-L, CMW-M and CMW-H, are developed to deal with respectively low-, middle- and high-level cross-modal information fusion. These modules use Depth-to-RGB Weighing (DW) and RGB-to-RGB Weighting (RW) to allow rich cross-modal and cross-scale interactions among feature layers generated by different network blocks. To effectively train the proposed Cross-Modal Weighting Network (CMWNet), we design a composite loss function that summarizes the errors between intermediate predictions and ground truth over different scales. With all these novel components working together, CMWNet effectively fuses information from RGB and depth channels, and meanwhile explores object localization and details across scales. Thorough evaluations demonstrate CMWNet consistently outperforms 15 state-of-the-art RGB-D SOD methods on seven popular benchmarks.
Reformulation of the No-Free-Lunch Theorem for Entangled Data Sets
Authors: Kunal Sharma, M. Cerezo, Zoë Holmes, Lukasz Cincio, Andrew Sornborger, Patrick J. Coles
Date: 2020-07-09 00:00:00
Tasks: BIG-bench Machine Learning, Learning Theory, Quantum Machine Learning

The no-free-lunch (NFL) theorem is a celebrated result in learning theory that limits one's ability to learn a function with a training data set. With the recent rise of quantum machine learning, it is natural to ask whether there is a quantum analog of the NFL theorem, which would restrict a quantum computer's ability to learn a unitary process (the quantum analog of a function) with quantum training data. However, in the quantum setting, the training data can possess entanglement, a strong correlation with no classical analog. In this work, we show that entangled data sets lead to an apparent violation of the (classical) NFL theorem. This motivates a reformulation that accounts for the degree of entanglement in the training set. As our main result, we prove a quantum NFL theorem whereby the fundamental limit on the learnability of a unitary is reduced by entanglement. We employ Rigetti's quantum computer to test both the classical and quantum NFL theorems. Our work establishes that entanglement is a commodity in quantum machine learning.
PIE-NET: Parametric Inference of Point Cloud Edges
Authors: Xiaogang Wang, Yuelang Xu, Kai Xu, Andrea Tagliasacchi, Bin Zhou, Ali Mahdavi-Amiri, Hao Zhang
Date: 2020-07-09 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Region Proposal

We introduce an end-to-end learnable technique to robustly identify feature edges in 3D point cloud data. We represent these edges as a collection of parametric curves (i.e.,lines, circles, and B-splines). Accordingly, our deep neural network, coined PIE-NET, is trained for parametric inference of edges. The network relies on a "region proposal" architecture, where a first module proposes an over-complete collection of edge and corner points, and a second module ranks each proposal to decide whether it should be considered. We train and evaluate our method on the ABC dataset, a large dataset of CAD models, and compare our results to those produced by traditional (non-learning) processing pipelines, as well as a recent deep learning based edge detector (EC-NET). Our results significantly improve over the state-of-the-art from both a quantitative and qualitative standpoint.
A Neuro-inspired Theory of Joint Human-Swarm Interaction
Authors: Jonas D. Hasbach, Maren Bennewitz
Date: 2020-07-09 00:00:00

Human-swarm interaction (HSI) is an active research challenge in the realms of swarm robotics and human-factors engineering. Here we apply a cognitive systems engineering perspective and introduce a neuro-inspired joint systems theory of HSI. The mindset defines predictions for adaptive, robust and scalable HSI dynamics and therefore has the potential to inform human-swarm loop design.
Multinomial Logit Bandit with Low Switching Cost
Authors: Kefan Dong, Yingkai Li, Qin Zhang, Yuan Zhou
Date: 2020-07-09 00:00:00
Proceeding: ICML 2020 1

We study multinomial logit bandit with limited adaptivity, where the algorithms change their exploration actions as infrequently as possible when achieving almost optimal minimax regret. We propose two measures of adaptivity: the assortment switching cost and the more fine-grained item switching cost. We present an anytime algorithm (AT-DUCB) with $O(N \log T)$ assortment switches, almost matching the lower bound $\Omega(\frac{N \log T}{ \log \log T})$. In the fixed-horizon setting, our algorithm FH-DUCB incurs $O(N \log \log T)$ assortment switches, matching the asymptotic lower bound. We also present the ESUCB algorithm with item switching cost $O(N \log^2 T)$.
CompRes: A Dataset for Narrative Structure in News
Authors: Effi Levi, Guy Mor, Shaul Shenhav, Tamir Sheafer
Date: 2020-07-09 00:00:00
Tasks: Articles

This paper addresses the task of automatically detecting narrative structures in raw texts. Previous works have utilized the oral narrative theory by Labov and Waletzky to identify various narrative elements in personal stories texts. Instead, we direct our focus to news articles, motivated by their growing social impact as well as their role in creating and shaping public opinion. We introduce CompRes -- the first dataset for narrative structure in news media. We describe the process in which the dataset was constructed: first, we designed a new narrative annotation scheme, better suited for news media, by adapting elements from the narrative theory of Labov and Waletzky (Complication and Resolution) and adding a new narrative element of our own (Success); then, we used that scheme to annotate a set of 29 English news articles (containing 1,099 sentences) collected from news and partisan websites. We use the annotated dataset to train several supervised models to identify the different narrative elements, achieving an $F_1$ score of up to 0.7. We conclude by suggesting several promising directions for future work.
Invertible Zero-Shot Recognition Flows
Authors: Yuming Shen, Jie Qin, Lei Huang
Date: 2020-07-09 00:00:00
Proceeding: ECCV 2020 8
Tasks: Zero-Shot Learning

Deep generative models have been successfully applied to Zero-Shot Learning (ZSL) recently. However, the underlying drawbacks of GANs and VAEs (e.g., the hardness of training with ZSL-oriented regularizers and the limited generation quality) hinder the existing generative ZSL models from fully bypassing the seen-unseen bias. To tackle the above limitations, for the first time, this work incorporates a new family of generative models (i.e., flow-based models) into ZSL. The proposed Invertible Zero-shot Flow (IZF) learns factorized data embeddings (i.e., the semantic factors and the non-semantic ones) with the forward pass of an invertible flow network, while the reverse pass generates data samples. This procedure theoretically extends conventional generative flows to a factorized conditional scheme. To explicitly solve the bias problem, our model enlarges the seen-unseen distributional discrepancy based on negative sample-based distance measurement. Notably, IZF works flexibly with either a naive Bayesian classifier or a held-out trainable one for zero-shot recognition. Experiments on widely-adopted ZSL benchmarks demonstrate the significant performance gain of IZF over existing methods, in both classic and generalized settings.
A Deep Joint Sparse Non-negative Matrix Factorization Framework for Identifying the Common and Subject-specific Functional Units of Tongue Motion During Speech
Authors: Jonghye Woo, Fangxu Xing, Jerry L. Prince, Maureen Stone, Arnold Gomez, Timothy G. Reese, Van J. Wedeen, Georges El Fakhri
Date: 2020-07-09 00:00:00
Tasks: Clustering

Intelligible speech is produced by creating varying internal local muscle groupings -- i.e., functional units -- that are generated in a systematic and coordinated manner. There are two major challenges in characterizing and analyzing functional units.~First, due to the complex and convoluted nature of tongue structure and function, it is of great importance to develop a method that can accurately decode complex muscle coordination patterns during speech. Second, it is challenging to keep identified functional units across subjects comparable due to their substantial variability. In this work, to address these challenges, we develop a new deep learning framework to identify common and subject-specific functional units of tongue motion during speech.~Our framework hinges on joint deep graph-regularized sparse non-negative matrix factorization (NMF) using motion quantities derived from displacements by tagged Magnetic Resonance Imaging. More specifically, we transform NMF with sparse and graph regularizations into modular architectures akin to deep neural networks by means of unfolding the Iterative Shrinkage-Thresholding Algorithm to learn interpretable building blocks and associated weighting map. We then apply spectral clustering to common and subject-specific weighting maps from which we jointly determine the common and subject-specific functional units. Experiments carried out with simulated datasets show that the proposed method achieved on par or better clustering performance over the comparison methods. Experiments carried out with in vivo tongue motion data show that the proposed method can determine the common and subject-specific functional units with increased interpretability and decreased size variability.
Attention or memory? Neurointerpretable agents in space and time
Authors: Lennart Bramlage, Aurelio Cortese
Date: 2020-07-09 00:00:00
Tasks: Atari Games, Dimensionality Reduction, Reinforcement Learning (RL)

In neuroscience, attention has been shown to bidirectionally interact with reinforcement learning (RL) processes. This interaction is thought to support dimensionality reduction of task representations, restricting computations to relevant features. However, it remains unclear whether these properties can translate into real algorithmic advantages for artificial agents, especially in dynamic environments. We design a model incorporating a self-attention mechanism that implements task-state representations in semantic feature-space, and test it on a battery of Atari games. To evaluate the agent's selective properties, we add a large volume of task-irrelevant features to observations. In line with neuroscience predictions, self-attention leads to increased robustness to noise compared to benchmark models. Strikingly, this self-attention mechanism is general enough, such that it can be naturally extended to implement a transient working-memory, able to solve a partially observable maze task. Lastly, we highlight the predictive quality of attended stimuli. Because we use semantic observations, we can uncover not only which features the agent elects to base decisions on, but also how it chooses to compile more complex, relational features from simpler ones. These results formally illustrate the benefits of attention in deep RL and provide evidence for the interpretability of self-attention mechanisms.
Identifying efficient controls of complex interaction networks using genetic algorithms
Authors: Victor-Bogdan Popescu, Krishna Kanhaiya, Iulian Năstac, Eugen Czeizler, Ion Petre
Date: 2020-07-09 00:00:00

Control theory has seen recently impactful applications in network science, especially in connections with applications in network medicine. A key topic of research is that of finding minimal external interventions that offer control over the dynamics of a given network, a problem known as network controllability. We propose in this article a new solution for this problem based on genetic algorithms. We tailor our solution for applications in computational drug repurposing, seeking to maximise its use of FDA-approved drug targets in a given disease-specific protein-protein interaction network. We show how our algorithm identifies a number of potentially efficient drugs for breast, ovarian, and pancreatic cancer. We demonstrate our algorithm on several benchmark networks from cancer medicine, social networks, electronic circuits, and several random networks with their edges distributed according to the Erd\H{o}s-R\'{e}nyi, the small-world, and the scale-free properties. Overall, we show that our new algorithm is more efficient in identifying relevant drug targets in a disease network, advancing the computational solutions needed for new therapeutic and drug repurposing approaches.
Guru, Partner, or Pencil Sharpener? Understanding Designers' Attitudes Towards Intelligent Creativity Support Tools
Authors: Angus Main, Mick Grierson
Date: 2020-07-09 00:00:00

Creativity Support Tools (CST) aim to enhance human creativity, but the deeply personal and subjective nature of creativity makes the design of universal support tools challenging. Individuals develop personal approaches to creativity, particularly in the context of commercial design where signature styles and techniques are valuable commodities. Artificial Intelligence (AI) and Machine Learning (ML) techniques could provide a means of creating 'intelligent' CST which learn and adapt to personal styles of creativity. Identifying what kind of role such tools could play in the design process requires a better understanding of designers' attitudes towards working with AI, and their willingness to include it in their personal creative process. This paper details the results of a survey of professional designers which indicates a positive and pragmatic attitude towards collaborating with AI tools, and a particular opportunity for incorporating them in the research stages of a design project.
Improving the Robustness of Trading Strategy Backtesting with Boltzmann Machines and Generative Adversarial Networks
Authors: Edmond Lezmi, Jules Roche, Thierry Roncalli, Jiali Xu
Date: 2020-07-09 00:00:00
Tasks: Management, Time Series, Time Series Analysis

This article explores the use of machine learning models to build a market generator. The underlying idea is to simulate artificial multi-dimensional financial time series, whose statistical properties are the same as those observed in the financial markets. In particular, these synthetic data must preserve the probability distribution of asset returns, the stochastic dependence between the different assets and the autocorrelation across time. The article proposes then a new approach for estimating the probability distribution of backtest statistics. The final objective is to develop a framework for improving the risk management of quantitative investment strategies, in particular in the space of smart beta, factor investing and alternative risk premia.
Towards Open-World Recommendation: An Inductive Model-based Collaborative Filtering Approach
Authors: Qitian Wu, Hengrui Zhang, Xiaofeng Gao, Junchi Yan, Hongyuan Zha
Date: 2020-07-09 00:00:00
Tasks: Collaborative Filtering, Matrix Completion, Recommendation Systems, Representation Learning

Recommendation models can effectively estimate underlying user interests and predict one's future behaviors by factorizing an observed user-item rating matrix into products of two sets of latent factors. However, the user-specific embedding factors can only be learned in a transductive way, making it difficult to handle new users on-the-fly. In this paper, we propose an inductive collaborative filtering framework that contains two representation models. The first model follows conventional matrix factorization which factorizes a group of key users' rating matrix to obtain meta latents. The second model resorts to attention-based structure learning that estimates hidden relations from query to key users and learns to leverage meta latents to inductively compute embeddings for query users via neural message passing. Our model enables inductive representation learning for users and meanwhile guarantees equivalent representation capacity as matrix factorization. Experiments demonstrate that our model achieves promising results for recommendation on few-shot users with limited training ratings and new unseen users which are commonly encountered in open-world recommender systems.
Fast Transformers with Clustered Attention
Authors: Apoorv Vyas, Angelos Katharopoulos, François Fleuret
Date: 2020-07-09 00:00:00
Proceeding: NeurIPS 2020 12
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), speech-recognition, Speech Recognition

Transformers have been proven a successful model for a variety of tasks in sequence modeling. However, computing the attention matrix, which is their key component, has quadratic complexity with respect to the sequence length, thus making them prohibitively expensive for large sequences. To address this, we propose clustered attention, which instead of computing the attention for every query, groups queries into clusters and computes attention just for the centroids. To further improve this approximation, we use the computed clusters to identify the keys with the highest attention per query and compute the exact key/query dot products. This results in a model with linear complexity with respect to the sequence length for a fixed number of clusters. We evaluate our approach on two automatic speech recognition datasets and show that our model consistently outperforms vanilla transformers for a given computational budget. Finally, we demonstrate that our model can approximate arbitrarily complex attention distributions with a minimal number of clusters by approximating a pretrained BERT model on GLUE and SQuAD benchmarks with only 25 clusters and no loss in performance.
A Global Stochastic Optimization Particle Filter Algorithm
Authors: Mathieu Gerber, Randal Douc
Date: 2020-07-09 00:00:00
Tasks: Stochastic Optimization

We introduce a new online algorithm for expected log-likelihood maximization in situations where the objective function is multi-modal and/or has saddle points, that we term G-PFSO. The key element underpinning G-PFSO is a probability distribution which (a) is shown to concentrate on the target parameter value as the sample size increases and (b) can be efficiently estimated by means of a standard particle filter algorithm. This distribution depends on a learning rate, where the faster the learning rate the quicker it concentrates on the desired element of the search space, but the less likely G-PFSO is to escape from a local optimum of the objective function. In order to achieve a fast convergence rate with a slow learning rate, G-PFSO exploits the acceleration property of averaging, well-known in the stochastic gradient literature. Considering several challenging estimation problems, the numerical experiments show that, with high probability, G-PFSO successfully finds the highest mode of the objective function and converges to its global maximizer at the optimal rate. While the focus of this work is expected log-likelihood maximization, the proposed methodology and its theory apply more generally for optimizing a function defined through an expectation.
Predicting Court Decisions for Alimony: Avoiding Extra-legal Factors in Decision made by Judges and Not Understandable AI Models
Authors: Fabrice Muhlenbach, Long Nguyen Phuoc, Isabelle Sayn
Date: 2020-07-09 00:00:00
Tasks: Decision Making

The advent of machine learning techniques has made it possible to obtain predictive systems that have overturned traditional legal practices. However, rather than leading to systems seeking to replace humans, the search for the determinants in a court decision makes it possible to give a better understanding of the decision mechanisms carried out by the judge. By using a large amount of court decisions in matters of divorce produced by French jurisdictions and by looking at the variables that allow to allocate an alimony or not, and to define its amount, we seek to identify if there may be extra-legal factors in the decisions taken by the judges. From this perspective, we present an explainable AI model designed in this purpose by combining a classification with random forest and a regression model, as a complementary tool to existing decision-making scales or guidelines created by practitioners.