pwc-archive/papers-with-abstracts huggingface dataset:
Computação Urbana da Teoria à Prática: Fundamentos, Aplicações e Desafios
Authors: Diego O. Rodrigues, Frances A. Santos, Geraldo P. Rocha Filho, Ademar T. Akabane, Raquel Cabral, Roger Immich, Wellington L. Junior, Felipe D. Cunha, Daniel L. Guidoni, Thiago H. Silva, Denis Rosário, Eduardo Cerqueira, Antonio A. F. Loureiro, Leandro A. Villas
Date: 2019-12-02 00:00:00
Tasks: Management

The growing of cities has resulted in innumerable technical and managerial challenges for public administrators such as energy consumption, pollution, urban mobility and even supervision of private and public spaces in an appropriate way. Urban Computing emerges as a promising paradigm to solve such challenges, through the extraction of knowledge, from a large amount of heterogeneous data existing in urban space. Moreover, Urban Computing correlates urban sensing, data management, and analysis to provide services that have the potential to improve the quality of life of the citizens of large urban centers. Consider this context, this chapter aims to present the fundamentals of Urban Computing and the steps necessary to develop an application in this area. To achieve this goal, the following questions will be investigated, namely: (i) What are the main research problems of Urban Computing?; (ii) What are the technological challenges for the implementation of services in Urban Computing?; (iii) What are the main methodologies used for the development of services in Urban Computing?; and (iv) What are the representative applications in this field?
Prediction and optimization of NaV1.7 inhibitors based on machine learning methods
Authors: Weikaixin Kong, Xinyu Tu, Zhengwei Xie, Zhuo Huang
Date: 2019-11-29 00:00:00
Tasks: BIG-bench Machine Learning

We used machine learning methods to predict NaV1.7 inhibitors and found the model RF-CDK that performed best on the imbalanced dataset. Using the RF-CDK model for screening drugs, we got effective compounds K1. We use the cell patch clamp method to verify K1. However, because the model evaluation method in this article is not comprehensive enough, there is still a lot of research work to be performed, such as comparison with other existing methods. The target protein has multiple active sites and requires our further research. We need more detailed models to consider this biological process and compare it with the current results, which is an error in this article. So we want to withdraw this article.
Anti-Alignments -- Measuring The Precision of Process Models and Event Logs
Authors: Thomas Chatain, Mathilde Boltenhagen, Josep Carmona
Date: 2019-11-28 00:00:00

Processes are a crucial artefact in organizations, since they coordinate the execution of activities so that products and services are provided. The use of models to analyse the underlying processes is a well-known practice. However, due to the complexity and continuous evolution of their processes, organizations need an effective way of analysing the relation between processes and models. Conformance checking techniques asses the suitability of a process model in representing an underlying process, observed through a collection of real executions. One important metric in conformance checking is to asses the precision of the model with respect to the observed executions, i.e., characterize the ability of the model to produce behavior unrelated to the one observed. In this paper we present the notion of anti-alignment as a concept to help unveiling runs in the model that may deviate significantly from the observed behavior. Using anti-alignments, a new metric for precision is proposed. In contrast to existing metrics, anti-alignment based precision metrics satisfy most of the required axioms highlighted in a recent publication. Moreover, a complexity analysis of the problem of computing anti-alignments is provided, which sheds light into the practicability of using anti-alignment to estimate precision. Experiments are provided that witness the validity of the concepts introduced in this paper.
Molecular Generative Model Based On Adversarially Regularized Autoencoder
Authors: Seung Hwan Hong, Jaechang Lim, Seongok Ryu, Woo Youn Kim
Date: 2019-11-13 00:00:00
Tasks: Generative Adversarial Network

Deep generative models are attracting great attention as a new promising approach for molecular design. All models reported so far are based on either variational autoencoder (VAE) or generative adversarial network (GAN). Here we propose a new type model based on an adversarially regularized autoencoder (ARAE). It basically uses latent variables like VAE, but the distribution of the latent variables is obtained by adversarial training like in GAN. The latter is intended to avoid both inappropriate approximation of posterior distribution in VAE and difficulty in handling discrete variables in GAN. Our benchmark study showed that ARAE indeed outperformed conventional models in terms of validity, uniqueness, and novelty per generated molecule. We also demonstrated successful conditional generation of drug-like molecules with ARAE for both cases of single and multiple properties control. As a potential real-world application, we could generate EGFR inhibitors sharing the scaffolds of known active molecules while satisfying drug-like conditions simultaneously.
Pre-Training of Deep Bidirectional Protein Sequence Representations with Structural Information
Authors: Seonwoo Min, Seunghyun Park, Siwon Kim, Hyun-Soo Choi, Byunghan Lee, Sungroh Yoon
Date: 2019-11-25 00:00:00
Tasks: Language Modeling, Language Modelling, Masked Language Modeling, Only Connect Walls Dataset Task 1 (Grouping)

Bridging the exponentially growing gap between the numbers of unlabeled and labeled protein sequences, several studies adopted semi-supervised learning for protein sequence modeling. In these studies, models were pre-trained with a substantial amount of unlabeled data, and the representations were transferred to various downstream tasks. Most pre-training methods solely rely on language modeling and often exhibit limited performance. In this paper, we introduce a novel pre-training scheme called PLUS, which stands for Protein sequence representations Learned Using Structural information. PLUS consists of masked language modeling and a complementary protein-specific pre-training task, namely same-family prediction. PLUS can be used to pre-train various model architectures. In this work, we use PLUS to pre-train a bidirectional recurrent neural network and refer to the resulting model as PLUS-RNN. Our experiment results demonstrate that PLUS-RNN outperforms other models of similar size solely pre-trained with the language modeling in six out of seven widely used protein biology tasks. Furthermore, we present the results from our qualitative interpretation analyses to illustrate the strengths of PLUS-RNN. PLUS provides a novel way to exploit evolutionary relationships among unlabeled proteins and is broadly applicable across a variety of protein biology tasks. We expect that the gap between the numbers of unlabeled and labeled proteins will continue to grow exponentially, and the proposed pre-training method will play a larger role.
CORE: Automatic Molecule Optimization Using Copy & Refine Strategy
Authors: Tianfan Fu, Cao Xiao, Jimeng Sun
Date: 2019-11-23 00:00:00

Molecule optimization is about generating molecule $Y$ with more desirable properties based on an input molecule $X$. The state-of-the-art approaches partition the molecules into a large set of substructures $S$ and grow the new molecule structure by iteratively predicting which substructure from $S$ to add. However, since the set of available substructures $S$ is large, such an iterative prediction task is often inaccurate especially for substructures that are infrequent in the training data. To address this challenge, we propose a new generating strategy called "Copy & Refine" (CORE), where at each step the generator first decides whether to copy an existing substructure from input $X$ or to generate a new substructure, then the most promising substructure will be added to the new molecule. Combining together with scaffolding tree generation and adversarial training, CORE can significantly improve several latest molecule optimization methods in various measures including drug likeness (QED), dopamine receptor (DRD2) and penalized LogP. We tested CORE and baselines using the ZINC database and CORE obtained up to 11% and 21% relatively improvement over the baselines on success rate on the complete test set and the subset with infrequent substructures, respectively.
Recurrent Neural Networks (RNNs): A gentle Introduction and Overview
Authors: Robin M. Schmidt
Date: 2019-11-23 00:00:00
Tasks: Language Modelling, speech-recognition, Speech Recognition

State-of-the-art solutions in the areas of "Language Modelling & Generating Text", "Speech Recognition", "Generating Image Descriptions" or "Video Tagging" have been using Recurrent Neural Networks as the foundation for their approaches. Understanding the underlying concepts is therefore of tremendous importance if we want to keep up with recent or upcoming publications in those areas. In this work we give a short overview over some of the most important concepts in the realm of Recurrent Neural Networks which enables readers to easily understand the fundamentals such as but not limited to "Backpropagation through Time" or "Long Short-Term Memory Units" as well as some of the more recent advances like the "Attention Mechanism" or "Pointer Networks". We also give recommendations for further reading regarding more complex topics where it is necessary.
Some observations concerning Off Training Set (OTS) error
Authors: Jonathan Baxter
Date: 2019-11-18 00:00:00
Tasks: BIG-bench Machine Learning

A form of generalisation error known as Off Training Set (OTS) error was recently introduced in [Wolpert, 1996b], along with a theorem showing that small training set error does not guarantee small OTS error, unless assumptions are made about the target function. Here it is shown that the applicability of this theorem is limited to models in which the distribution generating training data has no overlap with the distribution generating test data. It is argued that such a scenario is of limited relevance to machine learning.
Time Series Classification: Lessons Learned in the (Literal) Field while Studying Chicken Behavior
Authors: Alireza Abdoli, Amy C. Murillo, Alec C. Gerry, Eamonn J. Keogh
Date: 2019-11-21 00:00:00
Tasks: Clustering, General Classification, Time Series, Time Series Analysis, Time Series Classification

Poultry farms are a major contributor to the human food chain. However, around the world, there have been growing concerns about the quality of life for the livestock in poultry farms; and increasingly vocal demands for improved standards of animal welfare. Recent advances in sensing technologies and machine learning allow the possibility of monitoring birds, and employing the lessons learned to improve the welfare for all birds. This task superficially appears to be easy, yet, studying behavioral patterns involves collecting enormous amounts of data, justifying the term Big Data. Before the big data can be used for analytical purposes to tease out meaningful, well-conserved behavioral patterns, the collected data needs to be pre-processed. The pre-processing refers to processes for cleansing and preparing data so that it is in the format ready to be analyzed by downstream algorithms, such as classification and clustering algorithms. However, as we shall demonstrate, efficient pre-processing of chicken big data is both non-trivial and crucial towards success of further analytics.
Deep-Learning Estimation of Band Gap with the Reading-Periodic-Table Method and Periodic Convolution Layer
Authors: Tomohiko Konno
Date: 2019-11-16 00:00:00
Tasks: Band Gap, band gap classification, band gap regression, Deep Learning

We verified that the deep learning method named reading periodic table introduced by ref. Deep Learning Model for Finding New Superconductors, which utilizes deep learning to read the periodic table and the laws of the elements, is applicable not only for superconductors, for which the method was originally applied but also for other problems of materials by demonstrating band gap estimations. We then extended the method to learn the laws better by directly learning the cylindrical periodicity between the right- and left-most columns in the periodic table at the learning representation level, that is, by considering the left- and right-most columns to be adjacent to each other. Thus, while the original method handles the table as is, the extended method treats the periodic table as if its two edges are connected. This is achieved using novel layers named periodic convolution layers, which can handle inputs exhibiting periodicity and may be applied to other problems related to computer vision, time series, and so on for data that possess some periodicity. In the reading periodic table method, no material feature or descriptor is required as input. We demonstrated two types of deep learning estimation: methods to estimate the existence of a bandgap, and methods to estimate the value of the bandgap given when the existence of the bandgap in the materials is known. Finally, we discuss the limitations of the dataset and model evaluation method. We may be unable to distinguish good models based on the random train-test split scheme; thus, we must prepare an appropriate dataset where the training and test data are temporally separate. The code and data are open.
Applying ANN, ANFIS, and LSSVM Models for Estimation of Acid Solvent Solubility in Supercritical CO$_2$
Authors: Amin Bemani, Alireza Baghban, Shahaboddin Shamshirband, Amir Mosavi, Peter Csiba, Annamaria R. Varkonyi-Koczy
Date: 2019-11-21 00:00:00

In the present work, a novel and the robust computational investigation is carried out to estimate solubility of different acids in supercritical carbon dioxide. Four different algorithms such as radial basis function artificial neural network, Multi-layer Perceptron (MLP) artificial neural network (ANN), Least squares support vector machine (LSSVM) and adaptive neuro-fuzzy inference system (ANFIS) are developed to predict the solubility of different acids in carbon dioxide based on the temperature, pressure, hydrogen number, carbon number, molecular weight, and acid dissociation constant of acid. In the purpose of best evaluation of proposed models, different graphical and statistical analyses and also a novel sensitivity analysis are carried out. The present study proposed the great manners for best acid solubility estimation in supercritical carbon dioxide, which can be helpful for engineers and chemists to predict operational conditions in industries.
Characterizing the limits of human stability during motion: perturbative experiment validates a model-based approach for the Sit-to-Stand task
Authors: Patrick D. Holmes, Shannon M. Danforth, Xiao-Yu Fu, Talia Y. Moore, Ram Vasudevan
Date: 2019-08-05 00:00:00
Tasks: STS

Falls affect a growing number of the population each year. Clinical methods to identify those at greatest risk for falls usually evaluate individuals while they perform specific motions such as balancing or Sit-to-Stand (STS). Unfortunately these techniques have been shown to have poor predictive power and are unable to identify the magnitude, direction, and timing of perturbations that can cause an individual to lose stability during motion. To address this limitation, the recently proposed Stability Basin (SB) aims to characterize the set of perturbations that will cause an individual to fall under a specific motor control strategy. The SB is defined as the set of configurations that do not lead to failure for an individual under their chosen control strategy. This paper presents a novel method to compute the SB and the first experimental validation of the SB with an 11-person perturbative STS experiment involving forwards or backwards pulls from a motor-driven cable. The individually-constructed SBs are used to identify when a trial fails, i.e., when an individual must switch control strategies (indicated by a step or sit) to recover from a perturbation. The constructed SBs correctly predict the outcome of trials where failure was observed with over 90% accuracy, and correctly predict the outcome of successful trials with over 95% accuracy. The SB was compared to three other methods and was found to estimate the stable region with over 45% more accuracy in all cases. This study demonstrates that SBs offer a novel model-based approach for quantifying stability during motion, which could be used in physical therapy for individuals at risk of falling.
Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
Authors: Badr-Eddine Chérief-Abdellatif, Pierre Alquier
Date: 2019-12-12 00:00:00

Many works in statistics aim at designing a universal estimation procedure, that is, an estimator that would converge to the best approximation of the (unknown) data generating distribution in a model, without any assumption on this distribution. This question is of major interest, in particular because the universality property leads to the robustness of the estimator. In this paper, we tackle the problem of universal estimation using a minimum distance estimator presented in Briol et al. (2019) based on the Maximum Mean Discrepancy. We show that the estimator is robust to both dependence and to the presence of outliers in the dataset. Finally, we provide a theoretical study of the stochastic gradient descent algorithm used to compute the estimator, and we support our findings with numerical simulations. ** The proof of Proposition 4.4 in the published version contains a mistake. The mistake is fixed here (and the bound is actually improved by a factor 2). **
Recurrent Highway Networks with Grouped Auxiliary Memory
Authors: Wei Luo ; Feng Yu
Date: 2019-12-13 00:00:00
Proceeding: IEEE Access 2019 12
Tasks: image-classification, Image Classification, Language Modeling, Language Modelling, Sequential Image Classification, Stock Trend Prediction

Recurrent neural networks (RNNs) are challenging to train, let alone those with deep spatial structures. Architectures built upon highway connections such as Recurrent Highway Network (RHN) were developed to allow larger step-to-step transition depth, leading to more expressive models. However, problems that require capturing long-term dependencies still can not be well addressed by these models. Moreover, the ability to keep long-term memories tends to diminish when the spatial depth increases, since deeper structure may accelerate gradient vanishing. In this paper, we address these issues by proposing a novel RNN architecture based on RHN, namely the Recurrent Highway Network with Grouped Auxiliary Memory (GAM-RHN). The proposed architecture interconnects the RHN with a set of auxiliary memory units specifically for storing long-term information via reading and writing operations, which is analogous to Memory Augmented Neural Networks (MANNs). Experimental results on artificial long time lag tasks show that GAM-RHNs can be trained efficiently while being deep in both time and space. We also evaluate the proposed architecture on a variety of tasks, including language modeling, sequential image classification, and financial market forecasting. The potential of our approach is demonstrated by achieving state-of-the-art results on these tasks.
Deep neuroethology of a virtual rodent
Authors: Josh Merel, Diego Aldarondo, Jesse Marshall, Yuval Tassa, Greg Wayne, Bence Ölveczky
Date: 2019-11-21 00:00:00
Proceeding: ICLR 2020 1
Tasks: Deep Reinforcement Learning

Parallel developments in neuroscience and deep learning have led to mutually productive exchanges, pushing our understanding of real and artificial neural networks in sensory and cognitive systems. However, this interaction between fields is less developed in the study of motor control. In this work, we develop a virtual rodent as a platform for the grounded study of motor activity in artificial models of embodied control. We then use this platform to study motor activity across contexts by training a model to solve four complex tasks. Using methods familiar to neuroscientists, we describe the behavioral representations and algorithms employed by different layers of the network using a neuroethological approach to characterize motor activity relative to the rodent's behavior and goals. We find that the model uses two classes of representations which respectively encode the task-specific behavioral strategies and task-invariant behavioral kinematics. These representations are reflected in the sequential activity and population dynamics of neural subpopulations. Overall, the virtual rodent facilitates grounded collaborations between deep reinforcement learning and motor neuroscience.
SPIN: A High Speed, High Resolution Vision Dataset for Tracking and Action Recognition in Ping Pong
Authors: Steven Schwarcz, Peng Xu, David D'Ambrosio, Juhana Kangaspunta, Anelia Angelova, Huong Phan, Navdeep Jaitly
Date: 2019-12-13 00:00:00
Tasks: Action Recognition, Pose Estimation, Vocal Bursts Intensity Prediction

We introduce a new high resolution, high frame rate stereo video dataset, which we call SPIN, for tracking and action recognition in the game of ping pong. The corpus consists of ping pong play with three main annotation streams that can be used to learn tracking and action recognition models -- tracking of the ping pong ball and poses of humans in the videos and the spin of the ball being hit by humans. The training corpus consists of 53 hours of data with labels derived from previous models in a semi-supervised method. The testing corpus contains 1 hour of data with the same information, except that crowd compute was used to obtain human annotations of the ball position, from which ball spin has been derived. Along with the dataset we introduce several baseline models that were trained on this data. The models were specifically chosen to be able to perform inference at the same rate as the images are generated -- specifically 150 fps. We explore the advantages of multi-task training on this data, and also show interesting properties of ping pong ball trajectories that are derived from our observational data, rather than from prior physics models. To our knowledge this is the first large scale dataset of ping pong; we offer it to the community as a rich dataset that can be used for a large variety of machine learning and vision tasks such as tracking, pose estimation, semi-supervised and unsupervised learning and generative modeling.
WaLDORf: Wasteless Language-model Distillation On Reading-comprehension
Authors: James Yi Tian, Alexander P. Kreuzer, Pai-Hung Chen, Hans-Martin Will
Date: 2019-12-13 00:00:00
Tasks: Language Modeling, Language Modelling, model, Natural Language Understanding, Reading Comprehension

Transformer based Very Large Language Models (VLLMs) like BERT, XLNet and RoBERTa, have recently shown tremendous performance on a large variety of Natural Language Understanding (NLU) tasks. However, due to their size, these VLLMs are extremely resource intensive and cumbersome to deploy at production time. Several recent publications have looked into various ways to distil knowledge from a transformer based VLLM (most commonly BERT-Base) into a smaller model which can run much faster at inference time. Here, we propose a novel set of techniques which together produce a task-specific hybrid convolutional and transformer model, WaLDORf, that achieves state-of-the-art inference speed while still being more accurate than previous distilled models.