pwc-archive/papers-with-abstracts huggingface dataset:
Discordant Relaxations of Misspecified Models
Authors: Lixiong Li, Désiré Kédagni, Ismaël Mourifié
Date: 2020-12-21 00:00:00
Tasks: valid

In many set-identified models, it is difficult to obtain a tractable characterization of the identified set. Therefore, researchers often rely on non-sharp identification conditions, and empirical results are often based on an outer set of the identified set. This practice is often viewed as conservative yet valid because an outer set is always a superset of the identified set. However, this paper shows that when the model is refuted by the data, two sets of non-sharp identification conditions derived from the same model could lead to disjoint outer sets and conflicting empirical results. We provide a sufficient condition for the existence of such discordancy, which covers models characterized by conditional moment inequalities and the Artstein (1983) inequalities. We also derive sufficient conditions for the non-existence of discordant submodels, therefore providing a class of models for which constructing outer sets cannot lead to misleading interpretations. In the case of discordancy, we follow Masten and Poirier (2021) by developing a method to salvage misspecified models, but unlike them, we focus on discrete relaxations. We consider all minimum relaxations of a refuted model that restores data-consistency. We find that the union of the identified sets of these minimum relaxations is robust to detectable misspecifications and has an intuitive empirical interpretation.
Unsupervised in-distribution anomaly detection of new physics through conditional density estimation
Authors: George Stein, Uros Seljak, Biwei Dai
Date: 2020-12-21 00:00:00
Tasks: Anomaly Detection, Density Estimation

Anomaly detection is a key application of machine learning, but is generally focused on the detection of outlying samples in the low probability density regions of data. Here we instead present and motivate a method for unsupervised in-distribution anomaly detection using a conditional density estimator, designed to find unique, yet completely unknown, sets of samples residing in high probability density regions. We apply this method towards the detection of new physics in simulated Large Hadron Collider (LHC) particle collisions as part of the 2020 LHC Olympics blind challenge, and show how we detected a new particle appearing in only 0.08% of 1 million collision events. The results we present are our original blind submission to the 2020 LHC Olympics, where it achieved the state-of-the-art performance.
Fast Physical Activity Suggestions: Efficient Hyperparameter Learning in Mobile Health
Authors: Marianne Menictas, Sabina Tomkins, Susan Murphy
Date: 2020-12-21 00:00:00

Users can be supported to adopt healthy behaviors, such as regular physical activity, via relevant and timely suggestions on their mobile devices. Recently, reinforcement learning algorithms have been found to be effective for learning the optimal context under which to provide suggestions. However, these algorithms are not necessarily designed for the constraints posed by mobile health (mHealth) settings, that they be efficient, domain-informed and computationally affordable. We propose an algorithm for providing physical activity suggestions in mHealth settings. Using domain-science, we formulate a contextual bandit algorithm which makes use of a linear mixed effects model. We then introduce a procedure to efficiently perform hyper-parameter updating, using far less computational resources than competing approaches. Not only is our approach computationally efficient, it is also easily implemented with closed form matrix algebraic updates and we show improvements over state of the art approaches both in speed and accuracy of up to 99% and 56% respectively.
Explicitly Encouraging Low Fractional Dimensional Trajectories Via Reinforcement Learning
Authors: Sean Gillen, Katie Byl
Date: 2020-12-21 00:00:00
Tasks: Dimensionality Reduction, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

A key limitation in using various modern methods of machine learning in developing feedback control policies is the lack of appropriate methodologies to analyze their long-term dynamics, in terms of making any sort of guarantees (even statistically) about robustness. The central reasons for this are largely due to the so-called curse of dimensionality, combined with the black-box nature of the resulting control policies themselves. This paper aims at the first of these issues. Although the full state space of a system may be quite large in dimensionality, it is a common feature of most model-based control methods that the resulting closed-loop systems demonstrate dominant dynamics that are rapidly driven to some lower-dimensional sub-space within. In this work we argue that the dimensionality of this subspace is captured by tools from fractal geometry, namely various notions of a fractional dimension. We then show that the dimensionality of trajectories induced by model free reinforcement learning agents can be influenced adding a post processing function to the agents reward signal. We verify that the dimensionality reduction is robust to noise being added to the system and show that that the modified agents are more actually more robust to noise and push disturbances in general for the systems we examined.
Defence against adversarial attacks using classical and quantum-enhanced Boltzmann machines
Authors: Aidan Kehoe, Peter Wittek, Yanbo Xue, Alejandro Pozas-Kerstjens
Date: 2020-12-21 00:00:00

We provide a robust defence to adversarial attacks on discriminative algorithms. Neural networks are naturally vulnerable to small, tailored perturbations in the input data that lead to wrong predictions. On the contrary, generative models attempt to learn the distribution underlying a dataset, making them inherently more robust to small perturbations. We use Boltzmann machines for discrimination purposes as attack-resistant classifiers, and compare them against standard state-of-the-art adversarial defences. We find improvements ranging from 5% to 72% against attacks with Boltzmann machines on the MNIST dataset. We furthermore complement the training with quantum-enhanced sampling from the D-Wave 2000Q annealer, finding results comparable with classical techniques and with marginal improvements in some cases. These results underline the relevance of probabilistic methods in constructing neural networks and highlight a novel scenario of practical relevance where quantum computers, even with limited hardware capabilites, could provide advantages over classical computers. This work is dedicated to the memory of Peter Wittek.
A Distributional Approach to Controlled Text Generation
Authors: Muhammad Khalifa, Hady Elsahar, Marc Dymetman
Date: 2020-12-21 00:00:00
Proceeding: ICLR 2021 1
Tasks: Text Generation

We propose a Distributional Approach for addressing Controlled Text Generation from pre-trained Language Models (LMs). This approach permits to specify, in a single formal framework, both "pointwise" and "distributional" constraints over the target LM -- to our knowledge, the first model with such generality -- while minimizing KL divergence from the initial LM distribution. The optimal target distribution is then uniquely determined as an explicit EBM (Energy-Based Model) representation. From that optimal representation we then train a target controlled Autoregressive LM through an adaptive distributional variant of Policy Gradient. We conduct a first set of experiments over pointwise constraints showing the advantages of our approach over a set of baselines, in terms of obtaining a controlled LM balancing constraint satisfaction with divergence from the initial LM. We then perform experiments over distributional constraints, a unique feature of our approach, demonstrating its potential as a remedy to the problem of Bias in Language Models. Through an ablation study, we show the effectiveness of our adaptive technique for obtaining faster convergence. (Code available at https://github.com/naver/gdc)
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks
Authors: Quynh Nguyen, Marco Mondelli, Guido Montufar
Date: 2020-12-21 00:00:00
Tasks: Memorization

A recent line of work has analyzed the theoretical properties of deep neural networks via the Neural Tangent Kernel (NTK). In particular, the smallest eigenvalue of the NTK has been related to the memorization capacity, the global convergence of gradient descent algorithms and the generalization of deep nets. However, existing results either provide bounds in the two-layer setting or assume that the spectrum of the NTK matrices is bounded away from 0 for multi-layer networks. In this paper, we provide tight bounds on the smallest eigenvalue of NTK matrices for deep ReLU nets, both in the limiting case of infinite widths and for finite widths. In the finite-width setting, the network architectures we consider are fairly general: we require the existence of a wide layer with roughly order of $N$ neurons, $N$ being the number of data samples; and the scaling of the remaining layer widths is arbitrary (up to logarithmic factors). To obtain our results, we analyze various quantities of independent interest: we give lower bounds on the smallest singular value of hidden feature matrices, and upper bounds on the Lipschitz constant of input-output feature maps.
Smoothed Gaussian Mixture Models for Video Classification and Recommendation
Authors: Sirjan Kafle, Aman Gupta, Xue Xia, Ananth Sankar, Xi Chen, Di Wen, Liang Zhang
Date: 2020-12-17 00:00:00
Tasks: Action Recognition, Classification, General Classification, Video Classification

Cluster-and-aggregate techniques such as Vector of Locally Aggregated Descriptors (VLAD), and their end-to-end discriminatively trained equivalents like NetVLAD have recently been popular for video classification and action recognition tasks. These techniques operate by assigning video frames to clusters and then representing the video by aggregating residuals of frames with respect to the mean of each cluster. Since some clusters may see very little video-specific data, these features can be noisy. In this paper, we propose a new cluster-and-aggregate method which we call smoothed Gaussian mixture model (SGMM), and its end-to-end discriminatively trained equivalent, which we call deep smoothed Gaussian mixture model (DSGMM). SGMM represents each video by the parameters of a Gaussian mixture model (GMM) trained for that video. Low-count clusters are addressed by smoothing the video-specific estimates with a universal background model (UBM) trained on a large number of videos. The primary benefit of SGMM over VLAD is smoothing which makes it less sensitive to small number of training samples. We show, through extensive experiments on the YouTube-8M classification task, that SGMM/DSGMM is consistently better than VLAD/NetVLAD by a small but statistically significant margin. We also show results using a dataset created at LinkedIn to predict if a member will watch an uploaded video.
Disentangling images with Lie group transformations and sparse coding
Authors: Ho Yin Chau, Frank Qiu, Yubei Chen, Bruno Olshausen
Date: 2020-12-11 00:00:00

Discrete spatial patterns and their continuous transformations are two important regularities contained in natural signals. Lie groups and representation theory are mathematical tools that have been used in previous works to model continuous image transformations. On the other hand, sparse coding is an important tool for learning dictionaries of patterns in natural signals. In this paper, we combine these ideas in a Bayesian generative model that learns to disentangle spatial patterns and their continuous transformations in a completely unsupervised manner. Images are modeled as a sparse superposition of shape components followed by a transformation that is parameterized by n continuous variables. The shape components and transformations are not predefined, but are instead adapted to learn the symmetries in the data, with the constraint that the transformations form a representation of an n-dimensional torus. Training the model on a dataset consisting of controlled geometric transformations of specific MNIST digits shows that it can recover these transformations along with the digits. Training on the full MNIST dataset shows that it can learn both the basic digit shapes and the natural transformations such as shearing and stretching that are contained in this data.
Observement as Universal Measurement
Authors: David G. Green, Kerri Morgan, Marc Cheong
Date: 2020-12-07 00:00:00

Measurement theory is the cornerstone of science, but no equivalent theory underpins the huge volumes of non-numerical data now being generated. In this study, we show that replacing numbers with alternative mathematical models, such as strings and graphs, generalises traditional measurement to provide rigorous, formal systems (`observement') for recording and interpreting non-numerical data. Moreover, we show that these representations are already widely used and identify general classes of interpretive methodologies implicit in representations based on character strings and graphs (networks). This implies that a generalised concept of measurement has the potential to reveal new insights as well as deep connections between different fields of research.
Deep Gaussian Processes for geophysical parameter retrieval
Authors: Daniel Heestermans Svendsen, Pablo Morales-Álvarez, Rafael Molina, Gustau Camps-Valls
Date: 2020-12-07 00:00:00
Tasks: Gaussian Processes, Retrieval

This paper introduces deep Gaussian processes (DGPs) for geophysical parameter retrieval. Unlike the standard full GP model, the DGP accounts for complicated (modular, hierarchical) processes, provides an efficient solution that scales well to large datasets, and improves prediction accuracy over standard full and sparse GP models. We give empirical evidence of performance for estimation of surface dew point temperature from infrared sounding data.
Warped Gaussian Processes in Remote Sensing Parameter Estimation and Causal Inference
Authors: Anna Mateo-Sanchis, Jordi Muñoz-Marí, Adrián Pérez-Suay, Gustau Camps-Valls
Date: 2020-12-09 00:00:00
Tasks: Causal Inference, Gaussian Processes, parameter estimation, regression

This paper introduces warped Gaussian processes (WGP) regression in remote sensing applications. WGP models output observations as a parametric nonlinear transformation of a GP. The parameters of such prior model are then learned via standard maximum likelihood. We show the good performance of the proposed model for the estimation of oceanic chlorophyll content from multispectral data, vegetation parameters (chlorophyll, leaf area index, and fractional vegetation cover) from hyperspectral data, and in the detection of the causal direction in a collection of 28 bivariate geoscience and remote sensing causal problems. The model consistently performs better than the standard GP and the more advanced heteroscedastic GP model, both in terms of accuracy and more sensible confidence intervals.
Convolutional Neural Networks from Image Markers
Authors: Barbara C. Benato, Italos E. de Souza, Felipe L. Galvão, Alexandre X. Falcão
Date: 2020-12-15 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

A technique named Feature Learning from Image Markers (FLIM) was recently proposed to estimate convolutional filters, with no backpropagation, from strokes drawn by a user on very few images (e.g., 1-3) per class, and demonstrated for coconut-tree image classification. This paper extends FLIM for fully connected layers and demonstrates it on different image classification problems. The work evaluates marker selection from multiple users and the impact of adding a fully connected layer. The results show that FLIM-based convolutional neural networks can outperform the same architecture trained from scratch by backpropagation.
Enhance Convolutional Neural Networks with Noise Incentive Block
Authors: Menghan Xia, Yi Wang, Chu Han, Tien-Tsin Wong
Date: 2020-12-09 00:00:00
Tasks: Image Generation, Translation

As a generic modeling tool, Convolutional Neural Networks (CNNs) have been widely employed in image generation and translation tasks. However, when fed with a flat input, current CNN models may fail to generate vivid results due to the spatially shared convolution kernels. We call it the flatness degradation of CNNs. Unfortunately, such degradation is the greatest obstacles to generate a spatially-variant output from a flat input, which has been barely discussed in the previous literature. To tackle this problem, we propose a model agnostic solution, i.e. Noise Incentive Block (NIB), which serves as a generic plug-in for any CNN generation model. The key idea is to break the flat input condition while keeping the intactness of the original information. Specifically, the NIB perturbs the input data symmetrically with a noise map and reassembles them in the feature domain as driven by the objective function. Extensive experiments show that existing CNN models equipped with NIB survive from the flatness degradation and are able to generate visually better results with richer details in some specific image generation tasks given flat inputs, e.g. semantic image synthesis, data-hidden image generation, and deep neural dithering.
Machine Learning Algorithm for NLOS Millimeter Wave in 5G V2X Communication
Authors: Deepika Mohan, G. G. Md. Nawaz Ali, Peter Han Joo Chong
Date: 2020-12-16 00:00:00
Tasks: Autonomous Driving, BIG-bench Machine Learning

The 5G vehicle-to-everything (V2X) communication for autonomous and semi-autonomous driving utilizes the wireless technology for communication and the Millimeter Wave bands are widely implemented in this kind of vehicular network application. The main purpose of this paper is to broadcast the messages from the mmWave Base Station to vehicles at LOS (Line-of-sight) and NLOS (Non-LOS). Relay using Machine Learning (RML) algorithm is formulated to train the mmBS for identifying the blockages within its coverage area and broadcast the messages to the vehicles at NLOS using a LOS nodes as a relay. The transmission of information is faster with higher throughput and it covers a wider bandwidth which is reused, therefore when performing machine learning within the coverage area of mmBS most of the vehicles in NLOS can be benefited. A unique method of relay mechanism combined with machine learning is proposed to communicate with mobile nodes at NLOS.
Non-autoregressive electron flow generation for reaction prediction
Authors: Hangrui Bi, Hengyi Wang, Chence Shi, Jian Tang
Date: 2020-12-16 00:00:00
Tasks: Computational chemistry, Decoder, Prediction

Reaction prediction is a fundamental problem in computational chemistry. Existing approaches typically generate a chemical reaction by sampling tokens or graph edits sequentially, conditioning on previously generated outputs. These autoregressive generating methods impose an arbitrary ordering of outputs and prevent parallel decoding during inference. We devise a novel decoder that avoids such sequential generating and predicts the reaction in a Non-Autoregressive manner. Inspired by physical-chemistry insights, we represent edge edits in a molecule graph as electron flows, which can then be predicted in parallel. To capture the uncertainty of reactions, we introduce latent variables to generate multi-modal outputs. Following previous works, we evaluate our model on USPTO MIT dataset. Our model achieves both an order of magnitude lower inference latency, with state-of-the-art top-1 accuracy and comparable performance on Top-K sampling.
Deep Learning of Cell Classification using Microscope Images of Intracellular Microtubule Networks
Authors: Aleksei Shpilman, Dmitry Boikiy, Marina Polyakova, Daniel Kudenko, Anton Burakov, Elena Nadezhdina
Date: 2020-12-16 00:00:00
Tasks: Classification, General Classification, image-classification, Image Classification

Microtubule networks (MTs) are a component of a cell that may indicate the presence of various chemical compounds and can be used to recognize properties such as treatment resistance. Therefore, the classification of MT images is of great relevance for cell diagnostics. Human experts find it particularly difficult to recognize the levels of chemical compound exposure of a cell. Improving the accuracy with automated techniques would have a significant impact on cell therapy. In this paper we present the application of Deep Learning to MT image classification and evaluate it on a large MT image dataset of animal cells with three degrees of exposure to a chemical agent. The results demonstrate that the learned deep network performs on par or better at the corresponding cell classification task than human experts. Specifically, we show that the task of recognizing different levels of chemical agent exposure can be handled significantly better by the neural network than by human experts.
BERTChem-DDI : Improved Drug-Drug Interaction Prediction from text using Chemical Structure Information
Authors: Ishani Mondal
Date: 2020-12-21 00:00:00
Proceeding: AACL (knlp) 2020 12
Tasks: Relation Extraction

Traditional biomedical version of embeddings obtained from pre-trained language models have recently shown state-of-the-art results for relation extraction (RE) tasks in the medical domain. In this paper, we explore how to incorporate domain knowledge, available in the form of molecular structure of drugs, for predicting Drug-Drug Interaction from textual corpus. We propose a method, BERTChem-DDI, to efficiently combine drug embeddings obtained from the rich chemical structure of drugs along with off-the-shelf domain-specific BioBERT embedding-based RE architecture. Experiments conducted on the DDIExtraction 2013 corpus clearly indicate that this strategy improves other strong baselines architectures by 3.4\% macro F1-score.
Subword Sampling for Low Resource Word Alignment
Authors: Ehsaneddin Asgari, Masoud Jalili Sabet, Philipp Dufter, Christopher Ringlstetter, Hinrich Schütze
Date: 2020-12-21 00:00:00
Tasks: Bayesian Optimization, Machine Translation, Word Alignment

Annotation projection is an important area in NLP that can greatly contribute to creating language resources for low-resource languages. Word alignment plays a key role in this setting. However, most of the existing word alignment methods are designed for a high resource setting in machine translation where millions of parallel sentences are available. This amount reduces to a few thousands of sentences when dealing with low-resource languages failing the existing established IBM models. In this paper, we propose subword sampling-based alignment of text units. This method's hypothesis is that the aggregation of different granularities of text for certain language pairs can help word-level alignment. For certain languages for which gold-standard alignments exist, we propose an iterative Bayesian optimization framework to optimize selecting possible subwords from the space of possible subword representations of the source and target sentences. We show that the subword sampling method consistently outperforms word-level alignment on six language pairs: English-German, English-French, English-Romanian, English-Persian, English-Hindi, and English-Inuktitut. In addition, we show that the hyperparameters learned for certain language pairs can be applied to other languages at no supervision and consistently improve the alignment results. We observe that using $5K$ parallel sentences together with our proposed subword sampling approach, we obtain similar F1 scores to the use of $100K$'s of parallel sentences in existing word-level fast-align/eflomal alignment methods.