pwc-archive/papers-with-abstracts huggingface dataset:
Ensemble Machine Learning Methods for Modeling COVID19 Deaths
Authors: R. Bathwal, P. Chitta, K. Tirumala, V. Varadarajan
Date: 2020-10-04 00:00:00
Tasks: BIG-bench Machine Learning

Using a hybrid of machine learning and epidemiological approaches, we propose a novel data-driven approach in predicting US COVID-19 deaths at a county level. The model gives a more complete description of the daily death distribution, outputting quantile-estimates instead of mean deaths, where the model's objective is to minimize the pinball loss on deaths reported by the New York Times coronavirus county dataset. The resulting quantile estimates accurately forecast deaths at an individual-county level for a variable-length forecast period, and the approach generalizes well across different forecast period lengths. We won the Caltech-run modeling competition out of 50+ teams, and our aggregate is competitive with the best COVID-19 modeling systems (on root mean squared error).
Active Tuning
Authors: Sebastian Otte, Matthias Karlbauer, Martin V. Butz
Date: 2020-10-02 00:00:00
Tasks: Denoising, Time Series, Time Series Analysis, Time Series Denoising, Time Series Prediction

We introduce Active Tuning, a novel paradigm for optimizing the internal dynamics of recurrent neural networks (RNNs) on the fly. In contrast to the conventional sequence-to-sequence mapping scheme, Active Tuning decouples the RNN's recurrent neural activities from the input stream, using the unfolding temporal gradient signal to tune the internal dynamics into the data stream. As a consequence, the model output depends only on its internal hidden dynamics and the closed-loop feedback of its own predictions; its hidden state is continuously adapted by means of the temporal gradient resulting from backpropagating the discrepancy between the signal observations and the model outputs through time. In this way, Active Tuning infers the signal actively but indirectly based on the originally learned temporal patterns, fitting the most plausible hidden state sequence into the observations. We demonstrate the effectiveness of Active Tuning on several time series prediction benchmarks, including multiple super-imposed sine waves, a chaotic double pendulum, and spatiotemporal wave dynamics. Active Tuning consistently improves the robustness, accuracy, and generalization abilities of all evaluated models. Moreover, networks trained for signal prediction and denoising can be successfully applied to a much larger range of noise conditions with the help of Active Tuning. Thus, given a capable time series predictor, Active Tuning enhances its online signal filtering, denoising, and reconstruction abilities without the need for additional training.
A Machine Learning Framework for Computing the Most Probable Paths of Stochastic Dynamical Systems
Authors: Yang Li, Jinqiao Duan, Xianbin Liu
Date: 2020-10-01 00:00:00
Tasks: BIG-bench Machine Learning

The emergence of transition phenomena between metastable states induced by noise plays a fundamental role in a broad range of nonlinear systems. The computation of the most probable paths is a key issue to understand the mechanism of transition behaviors. Shooting method is a common technique for this purpose to solve the Euler-Lagrange equation for the associated action functional, while losing its efficacy in high-dimensional systems. In the present work, we develop a machine learning framework to compute the most probable paths in the sense of Onsager-Machlup action functional theory. Specifically, we reformulate the boundary value problem of Hamiltonian system and design a neural network to remedy the shortcomings of shooting method. The successful applications of our algorithms to several prototypical examples demonstrate its efficacy and accuracy for stochastic systems with both (Gaussian) Brownian noise and (non-Gaussian) L\'evy noise. This novel approach is effective in exploring the internal mechanisms of rare events triggered by random fluctuations in various scientific fields.
Classifying Songs with EEG
Authors: Prashant Lawhatre, Bharatesh R Shiraguppi, Esha Sharma, Krishna Prasad Miyapuram, Derek Lomas
Date: 2020-10-01 00:00:00
Tasks: BIG-bench Machine Learning, EEG, Electroencephalogram (EEG)

This research study aims to use machine learning methods to characterize the EEG response to music. Specifically, we investigate how resonance in the EEG response correlates with individual aesthetic enjoyment. Inspired by the notion of musical processing as resonance, we hypothesize that the intensity of an aesthetic experience is based on the degree to which a participants EEG entrains to the perceptual input. To test this and other hypotheses, we have built an EEG dataset from 20 subjects listening to 12 two minute-long songs in random order. After preprocessing and feature construction, we used this dataset to train and test multiple machine learning models.
High Definition image classification in Geoscience using Machine Learning
Authors: Yajun An, Zachary Golden, Tarka Wilcox, Renzhi Cao
Date: 2020-09-25 00:00:00
Tasks: BIG-bench Machine Learning, Classification, General Classification, image-classification, Image Classification, Vocal Bursts Intensity Prediction

High Definition (HD) digital photos taken with drones are widely used in the study of Geoscience. However, blurry images are often taken in collected data, and it takes a lot of time and effort to distinguish clear images from blurry ones. In this work, we apply Machine learning techniques, such as Support Vector Machine (SVM) and Neural Network (NN) to classify HD images in Geoscience as clear and blurry, and therefore automate data cleaning in Geoscience. We compare the results of classification based on features abstracted from several mathematical models. Some of the implementation of our machine learning tool is freely available at: https://github.com/zachgolden/geoai.
A Multiresolution 3D Morphable Face Model and Fitting Framework
Authors: Patrik Huber, Guosheng Hu, Rafael Tena, Pouria Mortazavian, Willem P. Koppen, William Christmas, Matthias Rätsch, Josef Kittler
Date: 2016-02-01 00:00:00
Tasks: 3D Face Reconstruction, Face Generation, Face Model, Face Recognition, Facial Landmark Detection, Head Pose Estimation, Pose Estimation

3D Morphable Face Models are a powerful tool in computer vision. They consists of a PCA model of face shape and colour information and allow to reconstruct a 3D face from a single 2D image. 3D Morphable Face Models are used for 3D head pose estimation, face analysis, face recognition, and, more recently, facial landmark detection and tracking. However, they are not as widely used as 2D methods - the process of building and using a 3D model is much more involved.
In this paper, we present the Surrey Face Model, a multi-resolution 3D Morphable Model that we make available to the public for non-commercial purposes. The model contains different mesh resolution levels and landmark point annotations as well as metadata for texture remapping. Accompanying the model is a lightweight open-source C++ library designed with simplicity and ease of integration as its foremost goals. In addition to basic functionality, it contains pose estimation and face frontalisation algorithms. With the tools presented in this paper, we aim to close two gaps. First, by offering different model resolution levels and fast fitting functionality, we enable the use of a 3D Morphable Model in time-critical applications like tracking. Second, the software library makes it easy for the community to adopt the 3D Morphable Face Model in their research, and it offers a public place for collaboration.
Website Classification Using Word Based Multiple N -Gram Models and Random Search Oriented Feature Parameters
Authors: Ashadullah Shawon, Syed Tauhid Zuhori, Firoz Mahmud, Md. Jamil-Ur Rahman
Date: 2018-12-21 00:00:00
Tasks: Classification, General Classification, Hyperparameter Optimization

Website classification is a convenient starting point for building an intelligent web browser and social networking sites that can understand the favorite categories of a user and also detect adult or harmful websites perfectly. Classifying the web sites using the information of the Uniform Resource Locator (URL) is an important and fast technique. A perfect result is needed for URL classification to make it usable in the real world applications. So we have proposed an improved approach for URL classification that is able to provide a better result. We have introduced the word-based multiple n-gram models for efficient feature extraction and multinomial distribution for Naive Bayes classifier under the Random Search pipeline for hyperparameter optimization that finds the best parameters of the URL features. The experimental result of our research is compared with the result of previous research works and we have shown a better result than the existing result. Our experimental result provides 88.77% in recall and 87.63% in F1-Score which is the best performance so far.
Spatio-temporal deformable convolution for compressed video quality enhancement
Authors: Jianing Deng
Date: 2020-04-03 00:00:00
Tasks: Video Enhancement, Video Restoration

Recent years have witnessed remarkable success of deep learning methods in quality enhancement for compressed video. To better explore temporal information, existing methods usually estimate optical flow for temporal motion compensation. However, since compressed video could be seriously distorted by various compression artifacts, the estimated optical flow tends to be inaccurate and unreliable, thereby resulting in ineffective quality enhancement. In addition, optical flow estimation for consecutive frames is generally conducted in a pairwise manner, which is computational expensive and inefficient. In this paper, we propose a fast yet effective method for compressed video quality enhancement by incorporating a novel Spatio-Temporal Deformable Fusion (STDF) scheme to aggregate temporal information. Specifically, the proposed STDF takes a target frame along with its neighboring reference frames as input to jointly predict an offset field to deform the spatio-temporal sampling positions of convolution. As a result, complementary information from both target and reference frames can be fused within a single Spatio-Temporal Deformable Convolution (STDC) operation. Extensive experiments show that our method achieves the state-of-the-art performance of compressed video quality enhancement in terms of both accuracy and efficiency.
Feature Selection on Lyme Disease Patient Survey Data
Authors: Joshua Vendrow, Jamie Haddock, Deanna Needell, Lorraine Johnson
Date: 2020-08-24 00:00:00
Tasks: BIG-bench Machine Learning, feature selection, Survey

Lyme disease is a rapidly growing illness that remains poorly understood within the medical community. Critical questions about when and why patients respond to treatment or stay ill, what kinds of treatments are effective, and even how to properly diagnose the disease remain largely unanswered. We investigate these questions by applying machine learning techniques to a large scale Lyme disease patient registry, MyLymeData, developed by the nonprofit LymeDisease.org. We apply various machine learning methods in order to measure the effect of individual features in predicting participants' answers to the Global Rating of Change (GROC) survey questions that assess the self-reported degree to which their condition improved, worsened, or remained unchanged following antibiotic treatment. We use basic linear regression, support vector machines, neural networks, entropy-based decision tree models, and $k$-nearest neighbors approaches. We first analyze the general performance of the model and then identify the most important features for predicting participant answers to GROC. After we identify the "key" features, we separate them from the dataset and demonstrate the effectiveness of these features at identifying GROC. In doing so, we highlight possible directions for future study both mathematically and clinically.
A Bayesian Model for Discovering Typological Implications
Authors: Hal Daumé III, Lyle Campbell
Date: 2009-07-04 00:00:00
Tasks: model

A standard form of analysis for linguistic typology is the universal implication. These implications state facts about the range of extant languages, such as ``if objects come after verbs, then adjectives come after nouns.'' Such implications are typically discovered by painstaking hand analysis over a small sample of languages. We propose a computational model for assisting at this process. Our model is able to discover both well-known implications as well as some novel implications that deserve further study. Moreover, through a careful application of hierarchical analysis, we are able to cope with the well-known sampling problem: languages are not independent.
Learning to Rank Using Localized Geometric Mean Metrics
Authors: Yuxin Su, Irwin King, Michael Lyu
Date: 2017-05-22 00:00:00
Tasks: Computational Efficiency, Learning-To-Rank, Metric Learning

Many learning-to-rank (LtR) algorithms focus on query-independent model, in which query and document do not lie in the same feature space, and the rankers rely on the feature ensemble about query-document pair instead of the similarity between query instance and documents. However, existing algorithms do not consider local structures in query-document feature space, and are fragile to irrelevant noise features. In this paper, we propose a novel Riemannian metric learning algorithm to capture the local structures and develop a robust LtR algorithm. First, we design a concept called \textit{ideal candidate document} to introduce metric learning algorithm to query-independent model. Previous metric learning algorithms aiming to find an optimal metric space are only suitable for query-dependent model, in which query instance and documents belong to the same feature space and the similarity is directly computed from the metric space. Then we extend the new and extremely fast global Geometric Mean Metric Learning (GMML) algorithm to develop a localized GMML, namely L-GMML. Based on the combination of local learned metrics, we employ the popular Normalized Discounted Cumulative Gain~(NDCG) scorer and Weighted Approximate Rank Pairwise (WARP) loss to optimize the \textit{ideal candidate document} for each query candidate set. Finally, we can quickly evaluate all candidates via the similarity between the \textit{ideal candidate document} and other candidates. By leveraging the ability of metric learning algorithms to describe the complex structural information, our approach gives us a principled and efficient way to perform LtR tasks. The experiments on real-world datasets demonstrate that our proposed L-GMML algorithm outperforms the state-of-the-art metric learning to rank methods and the stylish query-independent LtR algorithms regarding accuracy and computational efficiency.
TensorBNN: Bayesian Inference for Neural Networks using Tensorflow
Authors: Braden Kronheim, Michelle Kuchera, Harrison Prosper
Date: 2020-09-30 00:00:00
Tasks: Bayesian Inference, GPU

TensorBNN is a new package based on TensorFlow that implements Bayesian inference for modern neural network models. The posterior density of neural network model parameters is represented as a point cloud sampled using Hamiltonian Monte Carlo. The TensorBNN package leverages TensorFlow's architecture and training features as well as its ability to use modern graphics processing units (GPU) in both the training and prediction stages.
ivis Dimensionality Reduction Framework for Biomacromolecular Simulations
Authors: Hao Tian, Peng Tao
Date: 2020-04-22 00:00:00
Tasks: Dimensionality Reduction

Molecular dynamics (MD) simulations have been widely applied to study macromolecules including proteins. However, high-dimensionality of the datasets produced by simulations makes it difficult for thorough analysis, and further hinders a deeper understanding of biomacromolecules. To gain more insights into the protein structure-function relations, appropriate dimensionality reduction methods are needed to project simulations onto low-dimensional spaces. Linear dimensionality reduction methods, such as principal component analysis (PCA) and time-structure based independent component analysis (t-ICA), could not preserve sufficient structural information. Though better than linear methods, nonlinear methods, such as t-distributed stochastic neighbor embedding (t-SNE), still suffer from the limitations in avoiding system noise and keeping inter-cluster relations. ivis is a novel deep learning-based dimensionality reduction method originally developed for single-cell datasets. Here we applied this framework for the study of light, oxygen and voltage (LOV) domain of diatom Phaeodactylum tricornutum aureochrome 1a (PtAu1a). Compared with other methods, ivis is shown to be superior in constructing Markov state model (MSM), preserving information of both local and global distances and maintaining similarity between high dimension and low dimension with the least information loss. Moreover, ivis framework is capable of providing new prospective for deciphering residue-level protein allostery through the feature weights in the neural network. Overall, ivis is a promising member in the analysis toolbox for proteins.
Parameter estimation in FACS-seq enables high-throughput characterization of phenotypic heterogeneity
Authors: Huibao Feng, Chong Zhang
Date: 2020-10-07 00:00:00
Tasks: Diversity, parameter estimation

Phenotypic heterogeneity is a most fascinating property of a population of cells, which shows the differences among individuals even with the same genetic background and extracellular environmental conditions. However, the lack of high-throughput analysis of phenotypic diversity has limited our research progress. To deal with it, we constructed a novel parameter estimation method in FACS-seq, a commonly used experimental framework, to achieve simultaneous characterization of thousands of variants in a library. We further demonstrated the model's ability in estimating the expression properties of each variant, which we believe can help to decipher the mechanisms of phenotypic heterogeneity.
DeNERT-KG: Named Entity and Relation Extraction Model Using DQN, Knowledge Graph, and BERT
Authors: SungMin Yang, SoYeop Yoo, OkRan Jeong
Date: 2020-09-15 00:00:00
Tasks: Language Modeling, Language Modelling, named-entity-recognition, Named Entity Recognition, Named Entity Recognition (NER), NER, Object, Relation, Relation Extraction, Sentence

Along with studies on artificial intelligence technology, research is also being carried out actively in the field of natural language processing to understand and process people’s language, in other words, natural language. For computers to learn on their own, the skill of understanding natural language is very important. There are a wide variety of tasks involved in the field of natural language processing, but we would like to focus on the named entity registration and relation extraction task, which is considered to be the most important in understanding sentences. We propose DeNERT-KG, a model that can extract subject, object, and relationships, to grasp the meaning inherent in a sentence. Based on the BERT language model and Deep Q-Network, the named entity recognition (NER) model for extracting subject and object is established, and a knowledge graph is applied for relation extraction. Using the DeNERT-KG model, it is possible to extract the subject, type of subject, object, type of object, and relationship from a sentence, and verify this model through experiments.
Data-driven Thresholding in Denoising with Spectral Graph Wavelet Transform
Authors: Basile de Loynes, Fabien Navarro, Baptiste Olivier
Date: 2019-06-05 00:00:00
Tasks: Denoising, Unity

This paper is devoted to adaptive signal denoising in the context of Graph Signal Processing (GSP) using Spectral Graph Wavelet Transform (SGWT). This issue is addressed \emph{via} a data-driven thresholding process in the transformed domain by optimizing the parameters in the sense of the Mean Square Error (MSE) using the Stein's Unbiased Risk Estimator (SURE). The SGWT considered is built upon a partition of unity making the transform semi-orthogonal so that the optimization can be performed in the transformed domain. However, since the SGWT is over-complete, the divergence term in the SURE needs to be computed in the context of correlated noise. Two thresholding strategies called coordinatewise and block thresholding process are investigated. For each of them, the SURE is derived for a whole family of elementary thresholding functions among which the soft threshold and the James-Stein threshold. This multi-scales analysis shows better performance than the most recent methods from the literature. That is illustrated numerically for a series of signals on different graphs.
Localized Fourier Analysis for Graph Signal Processing
Authors: Basile de Loynes, Fabien Navarro, Baptiste Olivier
Date: 2019-06-11 00:00:00
Tasks: Denoising

We propose a new point of view in the study of Fourier analysis on graphs, taking advantage of localization in the Fourier domain. For a signal $f$ on vertices of a weighted graph $\mathcal{G}$ with Laplacian matrix $\mathcal{L}$, standard Fourier analysis of $f$ relies on the study of functions $g(\mathcal{L})f$ for some filters $g$ on $I_\mathcal{L}$, the smallest interval containing the Laplacian spectrum ${\mathrm sp}(\mathcal{L}) \subset I_\mathcal{L}$. We show that for carefully chosen partitions $I_\mathcal{L} = \sqcup_{1\leq k\leq K} I_k$ ($I_k \subset I_\mathcal{L}$), there are many advantages in understanding the collection $(g(\mathcal{L}_{I_k})f)_{1\leq k\leq K}$ instead of $g(\mathcal{L})f$ directly, where $\mathcal{L}_I$ is the projected matrix $P_I(\mathcal{L})\mathcal{L}$. First, the partition provides a convenient modelling for the study of theoretical properties of Fourier analysis and allows for new results in graph signal analysis (\emph{e.g.} noise level estimation, Fourier support approximation). We extend the study of spectral graph wavelets to wavelets localized in the Fourier domain, called LocLets, and we show that well-known frames can be written in terms of LocLets. From a practical perspective, we highlight the interest of the proposed localized Fourier analysis through many experiments that show significant improvements in two different tasks on large graphs, noise level estimation and signal denoising. Moreover, efficient strategies permit to compute sequence $(g(\mathcal{L}_{I_k})f)_{1\leq k\leq K}$ with the same time complexity as for the computation of $g(\mathcal{L})f$.
HetSeq: Distributed GPU Training on Heterogeneous Infrastructure
Authors: Yifan Ding, Nicholas Botzer, Tim Weninger
Date: 2020-09-25 00:00:00
Tasks: GPU, image-classification, Image Classification, Language Modeling, Language Modelling, Translation

Modern deep learning systems like PyTorch and Tensorflow are able to train enormous models with billions (or trillions) of parameters on a distributed infrastructure. These systems require that the internal nodes have the same memory capacity and compute performance. Unfortunately, most organizations, especially universities, have a piecemeal approach to purchasing computer systems resulting in a heterogeneous infrastructure, which cannot be used to compute large models. The present work describes HetSeq, a software package adapted from the popular PyTorch package that provides the capability to train large neural network models on heterogeneous infrastructure. Experiments with transformer translation and BERT language model shows that HetSeq scales over heterogeneous systems. HetSeq can be easily extended to other models like image classification. Package with supported document is publicly available at https://github.com/yifding/hetseq.