pwc-archive/papers-with-abstracts huggingface dataset:
Earthmover-based manifold learning for analyzing molecular conformation spaces
Authors: Nathan Zelesko, Amit Moscovich, Joe Kileel, Amit Singer
Date: 2019-10-16 00:00:00
Tasks: Dimensionality Reduction

In this paper, we propose a novel approach for manifold learning that combines the Earthmover's distance (EMD) with the diffusion maps method for dimensionality reduction. We demonstrate the potential benefits of this approach for learning shape spaces of proteins and other flexible macromolecules using a simulated dataset of 3-D density maps that mimic the non-uniform rotary motion of ATP synthase. Our results show that EMD-based diffusion maps require far fewer samples to recover the intrinsic geometry than the standard diffusion maps algorithm that is based on the Euclidean distance. To reduce the computational burden of calculating the EMD for all volume pairs, we employ a wavelet-based approximation to the EMD which reduces the computation of the pairwise EMDs to a computation of pairwise weighted-$\ell_1$ distances between wavelet coefficient vectors.
Response to NITRD, NCO, NSF Request for Information on "Update to the 2016 National Artificial Intelligence Research and Development Strategic Plan"
Authors: J. Amundson, J. Annis, C. Avestruz, D. Bowring, J. Caldeira, G. Cerati, C. Chang, S. Dodelson, D. Elvira, A. Farahi, K. Genser, L. Gray, O. Gutsche, P. Harris, J. Kinney, J. B. Kowalkowski, R. Kutschke, S. Mrenna, B. Nord, A. Para, K. Pedro, G. N. Perdue, A. Scheinker, P. Spentzouris, J. St. John, N. Tran, S. Trivedi, L. Trouille, W. L. K. Wu, C. R. Bom
Date: 2019-11-05 00:00:00

We present a response to the 2018 Request for Information (RFI) from the NITRD, NCO, NSF regarding the "Update to the 2016 National Artificial Intelligence Research and Development Strategic Plan." Through this document, we provide a response to the question of whether and how the National Artificial Intelligence Research and Development Strategic Plan (NAIRDSP) should be updated from the perspective of Fermilab, America's premier national laboratory for High Energy Physics (HEP). We believe the NAIRDSP should be extended in light of the rapid pace of development and innovation in the field of Artificial Intelligence (AI) since 2016, and present our recommendations below. AI has profoundly impacted many areas of human life, promising to dramatically reshape society --- e.g., economy, education, science --- in the coming years. We are still early in this process. It is critical to invest now in this technology to ensure it is safe and deployed ethically. Science and society both have a strong need for accuracy, efficiency, transparency, and accountability in algorithms, making investments in scientific AI particularly valuable. Thus far the US has been a leader in AI technologies, and we believe as a national Laboratory it is crucial to help maintain and extend this leadership. Moreover, investments in AI will be important for maintaining US leadership in the physical sciences.
HighRes-net: Multi-Frame Super-Resolution by Recursive Fusion
Authors: Michel Deudon, Alfredo Kalaitzis, Md Rifat Arefin, Israel Goytom, Zhichao Lin, Kris Sankaran, Vincent Michalski, Samira E. Kahou, Julien Cornebise, Yoshua Bengio
Date: 2020-01-01 00:00:00
Proceeding: ICLR 2020 1
Tasks: De-aliasing, Earth Observation, Image Registration, Image Super-Resolution, Multi-Frame Super-Resolution, Super-Resolution

Generative deep learning has sparked a new wave of Super-Resolution (SR) algorithms that enhance single images with impressive aesthetic results, albeit with imaginary details. Multi-frame Super-Resolution (MFSR) offers a more grounded approach to the ill-posed problem, by conditioning on multiple low-resolution views. This is important for satellite monitoring of human impact on the planet -- from deforestation, to human rights violations -- that depend on reliable imagery. To this end, we present HighRes-net, the first deep learning approach to MFSR that learns its sub-tasks in an end-to-end fashion: (i) co-registration, (ii) fusion, (iii) up-sampling, and (iv) registration-at-the-loss. Co-registration of low-res views is learned implicitly through a reference-frame channel, with no explicit registration mechanism. We learn a global fusion operator that is applied recursively on an arbitrary number of low-res pairs. We introduce a registered loss, by learning to align the SR output to a ground-truth through ShiftNet. We show that by learning deep representations of multiple views, we can super-resolve low-resolution signals and enhance Earth observation data at scale. Our approach recently topped the European Space Agency's MFSR competition on real-world satellite imagery.
Random Forest Estimation of the Ordered Choice Model
Authors: Michael Lechner, Gabriel Okasa
Date: 2019-07-04 00:00:00
Tasks: model

In this paper we develop a new machine learning estimator for ordered choice models based on the random forest. The proposed Ordered Forest flexibly estimates the conditional choice probabilities while taking the ordering information explicitly into account. In addition to common machine learning estimators, it enables the estimation of marginal effects as well as conducting inference and thus provides the same output as classical econometric estimators. An extensive simulation study reveals a good predictive performance, particularly in settings with non-linearities and near-multicollinearity. An empirical application contrasts the estimation of marginal effects and their standard errors with an ordered logit model. A software implementation of the Ordered Forest is provided both in R and Python in the package orf available on CRAN and PyPI, respectively.
Wavenilm: A causal neural network for power disaggregation from the complex power signal
Authors: Alon Harell, Stephen Makonin, Ivan V. Bajić
Date: 2019-02-23 00:00:00
Tasks: Non-Intrusive Load Monitoring

Non-intrusive load monitoring (NILM) helps meet energy conservation goals by estimating individual appliance power usage from a single aggregate measurement. Deep neural networks have become increasingly popular in attempting to solve NILM problems; however, many of them are not causal which is important for real-time application. We present a causal 1-D convolutional neural network inspired by WaveNet for NILM on low-frequency data. We also study using various components of the complex power signal for NILM, and demonstrate that using all four components available in a popular NILM dataset (current, active power, reactive power, and apparent power) we achieve faster convergence and higher performance than state-of-the-art results for the same dataset.
ENCASE: An ENsemble ClASsifiEr for ECG classification using expert features and deep neural networks
Authors: Shenda Hong, Meng Wu, Yuxi Zhou, Qingyun Wang, Junyuan Shang, Hongyan Li, Junqing Xie
Date: 2017-09-24 00:00:00
Proceeding: 2017 Computing in Cardiology (CinC) 2017 9
Tasks: Arrhythmia Detection, Classification, ECG Classification, General Classification, Time Series Classification

We propose ENCASE to combine expert features and DNNs (Deep Neural Networks) together for ECG classification. We first explore and implement expert features from statistical area, signal processing area and medical area. Then, we build DNNs to automatically extract deep features. Besides, we propose a new algorithm to find the most representative wave (called centerwave) among long ECG record, and extract features from centerwave. Finally, we combine these features together and put them into ensemble classifiers. Experiment on 4-class ECG data classification reports 0.84 F1 score, which is much better than any of the single model.
Bounds on Multi-asset Derivatives via Neural Networks
Authors: Luca De Gennaro Aquino, Carole Bernard
Date: 2019-11-13 00:00:00
Tasks: Math

Using neural networks, we compute bounds on the prices of multi-asset derivatives given information on prices of related payoffs. As a main example, we focus on European basket options and include information on the prices of other similar options, such as spread options and/or basket options on subindices. We show that, in most cases, adding further constraints gives rise to bounds that are considerably tighter and discuss the maximizing/minimizing copulas achieving such bounds. Our approach follows the literature on constrained optimal transport and, in particular, builds on a recent paper by Eckstein and Kupper (2019, Appl. Math. Optim.).
A CONTEXTUALIZED, DIFFERENTIAL SEQUENCE MINING METHOD TO DERIVE STUDENTS' LEARNING BEHAVIOR PATTERNS
Authors: John S. Kinnebrew, Kirk M. Loretz, Gautam Biswas
Date: 2013-03-01 00:00:00
Proceeding: Kinnebrew, J. S., Loretz, K. M., & Biswas, G. (2013). A Contextualized, Differential Sequence Mining Method to Derive Students’ Learning Behavior Patterns. JEDM | Journal of Educational Data Mining, 5(1), 190-219. Retrieved from 2013 3

Computer-based learning environments can produce a wealth of data on student learning interactions. This paper presents an exploratory data mining methodology for assessing and comparing students' learning behaviors from these interaction traces. The core algorithm employs a novel combination of sequence mining techniques to identify differentially frequent patterns between groups of students (e.g., experimental versus control conditions or high versus low performers). We extend this technique by contextualizing the sequence mining with information about the stu- dent's performance over the course of the learning interactions. Specifically, we employ a piecewise linear segmentation algorithm in concert with the differential sequence mining technique to identify and compare segments of students' productive and unproductive learning behaviors. We present the results from the application of this exploratory data mining methodology to learning interaction trace data gathered during a recent middle school class study with the Betty's Brain learning environment. These results illustrate the potential of this methodology in identifying learning behavior patterns relevant to the investigation of metacognition and strategy use.
Non-Monotone Submodular Maximization with Multiple Knapsacks in Static and Dynamic Settings
Authors: Vanja Doskoč, Tobias Friedrich, Andreas Göbel, Frank Neumann, Aneta Neumann, Francesco Quinzan
Date: 2019-11-15 00:00:00
Tasks: Video Summarization

We study the problem of maximizing a non-monotone submodular function under multiple knapsack constraints. We propose a simple discrete greedy algorithm to approach this problem, and prove that it yields strong approximation guarantees for functions with bounded curvature. In contrast to other heuristics, this requires no problem relaxation to continuous domains and it maintains a constant-factor approximation guarantee in the problem size. In the case of a single knapsack, our analysis suggests that the standard greedy can be used in non-monotone settings. Additionally, we study this problem in a dynamic setting, by which knapsacks change during the optimization process. We modify our greedy algorithm to avoid a complete restart at each constraint update. This modification retains the approximation guarantees of the static case. We evaluate our results experimentally on a video summarization and sensor placement task. We show that our proposed algorithm competes with the state-of-the-art in static settings. Furthermore, we show that in dynamic settings with tight computational time budget, our modified greedy yields significant improvements over starting the greedy from scratch, in terms of the solution quality achieved.
Data Efficient Stagewise Knowledge Distillation
Authors: Akshay Kulkarni, Navid Panchi, Sharath Chandra Raparthy, Shital Chiddarwar
Date: 2019-11-15 00:00:00
Tasks: Knowledge Distillation, Model Compression, Quantization, Semantic Segmentation

Despite the success of Deep Learning (DL), the deployment of modern DL models requiring large computational power poses a significant problem for resource-constrained systems. This necessitates building compact networks that reduce computations while preserving performance. Traditional Knowledge Distillation (KD) methods that transfer knowledge from teacher to student (a) use a single-stage and (b) require the whole data set while distilling the knowledge to the student. In this work, we propose a new method called Stagewise Knowledge Distillation (SKD) which builds on traditional KD methods by progressive stagewise training to leverage the knowledge gained from the teacher, resulting in data-efficient distillation process. We evaluate our method on classification and semantic segmentation tasks. We show, across the tested tasks, significant performance gains even with a fraction of the data used in distillation, without compromising on the metric. We also compare our method with existing KD techniques and show that SKD outperforms them. Moreover, our method can be viewed as a generalized model compression technique that complements other model compression methods such as quantization or pruning.
TinyCNN: A Tiny Modular CNN Accelerator for Embedded FPGA
Authors: Ali Jahanshahi
Date: 2019-11-15 00:00:00
Tasks: General Classification, image-classification, Image Classification

In recent years, Convolutional Neural Network (CNN) based methods have achieved great success in a large number of applications and have been among the most powerful and widely used techniques in computer vision. However, CNN-based methods are computational-intensive and resource-consuming, and thus are hard to be integrated into embedded systems such as smart phones, smart glasses, and robots. FPGA is one of the most promising platforms for accelerating CNN, but the limited on-chip memory size limit the performance of FPGA accelerator for CNN. In this paper, we propose a framework for designing CNN accelerator on embedded FPGA for image classification. The proposed framework provides a tool for FPGA resource-aware design space exploration of CNNs and automatically generates the hardware description of the CNN to be programmed on a target FPGA. The framework consists of three main backends; software, hardware generation, and simulation/precision adjustment. The software backend serves as an API to the designer to design the CNN and train it according to the hardware resources that are available. Using the CNN model, hardware backend generates the necessary hardware components and integrates them to generate the hardware description of the CNN. Finaly, Simulation/precision adjustment backend adjusts the inter-layer precision units to minimize the classification error. We used 16-bit fixed-point data in a CNN accelerator (FPGA) and compared it to the exactly similar software version running on an ARM processor (32-bit floating point data). We encounter about 3% accuracy loss in classification of the accelerated (FPGA) version. In return, we got up to 15.75x speedup by classifying with the accelerated version on the FPGA.
Clustering of solutions in the symmetric binary perceptron
Authors: Carlo Baldassi, Riccardo Della Vecchia, Carlo Lucibello, Riccardo Zecchina
Date: 2019-11-15 00:00:00
Tasks: Clustering

The geometrical features of the (non-convex) loss landscape of neural network models are crucial in ensuring successful optimization and, most importantly, the capability to generalize well. While minimizers' flatness consistently correlates with good generalization, there has been little rigorous work in exploring the condition of existence of such minimizers, even in toy models. Here we consider a simple neural network model, the symmetric perceptron, with binary weights. Phrasing the learning problem as a constraint satisfaction problem, the analogous of a flat minimizer becomes a large and dense cluster of solutions, while the narrowest minimizers are isolated solutions. We perform the first steps toward the rigorous proof of the existence of a dense cluster in certain regimes of the parameters, by computing the first and second moment upper bounds for the existence of pairs of arbitrarily close solutions. Moreover, we present a non rigorous derivation of the same bounds for sets of $y$ solutions at fixed pairwise distances.
Unsupervised Attributed Multiplex Network Embedding
Authors: Chanyoung Park, Donghyun Kim, Jiawei Han, Hwanjo Yu
Date: 2019-11-15 00:00:00
Tasks: Network Embedding, Relation

Nodes in a multiplex network are connected by multiple types of relations. However, most existing network embedding methods assume that only a single type of relation exists between nodes. Even for those that consider the multiplexity of a network, they overlook node attributes, resort to node labels for training, and fail to model the global properties of a graph. We present a simple yet effective unsupervised network embedding method for attributed multiplex network called DMGI, inspired by Deep Graph Infomax (DGI) that maximizes the mutual information between local patches of a graph, and the global representation of the entire graph. We devise a systematic way to jointly integrate the node embeddings from multiple graphs by introducing 1) the consensus regularization framework that minimizes the disagreements among the relation-type specific node embeddings, and 2) the universal discriminator that discriminates true samples regardless of the relation types. We also show that the attention mechanism infers the importance of each relation type, and thus can be useful for filtering unnecessary relation types as a preprocessing step. Extensive experiments on various downstream tasks demonstrate that DMGI outperforms the state-of-the-art methods, even though DMGI is fully unsupervised.
Towards Personalized Dialog Policies for Conversational Skill Discovery
Authors: Maryam Fazel-Zarandi, Sampat Biswas, Ryan Summers, Ahmed Elmalt, Andy McCraw, Michael McPhilips, John Peach
Date: 2019-11-15 00:00:00
Tasks: Reinforcement Learning

Many businesses and consumers are extending the capabilities of voice-based services such as Amazon Alexa, Google Home, Microsoft Cortana, and Apple Siri to create custom voice experiences (also known as skills). As the number of these experiences increases, a key problem is the discovery of skills that can be used to address a user's request. In this paper, we focus on conversational skill discovery and present a conversational agent which engages in a dialog with users to help them find the skills that fulfill their needs. To this end, we start with a rule-based agent and improve it by using reinforcement learning. In this way, we enable the agent to adapt to different user attributes and conversational styles as it interacts with users. We evaluate our approach in a real production setting by deploying the agent to interact with real users, and show the effectiveness of the conversational agent in helping users find the skills that serve their request.
Penalized k-means algorithms for finding the correct number of clusters in a dataset
Authors: Behzad Kamgar-Parsi, Behrooz Kamgar-Parsi
Date: 2019-11-15 00:00:00

In many applications we want to find the number of clusters in a dataset. A common approach is to use the penalized k-means algorithm with an additive penalty term linear in the number of clusters. An open problem is estimating the value of the coefficient of the penalty term. Since estimating the value of the coefficient in a principled manner appears to be intractable for general clusters, we investigate "ideal clusters", i.e. identical spherical clusters with no overlaps and no outlier background noise. In this paper: (a) We derive, for the case of ideal clusters, rigorous bounds for the coefficient of the additive penalty. Unsurprisingly, the bounds depend on the correct number of clusters, which we want to find in the first place. We further show that additive penalty, even for this simplest case of ideal clusters, typically produces a weak and often ambiguous signature for the correct number of clusters. (b) As an alternative, we examine the k-means with multiplicative penalty, and show that this parameter-free formulation has a stronger, and less often ambiguous, signature for the correct number of clusters. We also empirically investigate certain types of deviations from ideal cluster assumption and show that combination of k-means with additive and multiplicative penalties can resolve ambiguous solutions.
Infinite-Horizon Policy-Gradient Estimation
Authors: Jonathan Baxter, Peter L. Bartlett
Date: 2011-06-03 00:00:00
Tasks: Reinforcement Learning

Gradient-based approaches to direct policy search in reinforcement learning have received much recent attention as a means to solve problems of partial observability and to avoid some of the problems associated with policy degradation in value-function methods. In this paper we introduce GPOMDP, a simulation-based algorithm for generating a {\em biased} estimate of the gradient of the {\em average reward} in Partially Observable Markov Decision Processes (POMDPs) controlled by parameterized stochastic policies. A similar algorithm was proposed by Kimura, Yamamura, and Kobayashi (1995). The algorithm's chief advantages are that it requires storage of only twice the number of policy parameters, uses one free parameter $\beta\in [0,1)$ (which has a natural interpretation in terms of bias-variance trade-off), and requires no knowledge of the underlying state. We prove convergence of GPOMDP, and show how the correct choice of the parameter $\beta$ is related to the {\em mixing time} of the controlled POMDP. We briefly describe extensions of GPOMDP to controlled Markov chains, continuous state, observation and control spaces, multiple-agents, higher-order derivatives, and a version for training stochastic policies with internal states. In a companion paper (Baxter, Bartlett, & Weaver, 2001) we show how the gradient estimates generated by GPOMDP can be used in both a traditional stochastic gradient algorithm and a conjugate-gradient procedure to find local optima of the average reward
A nonparametric framework for inferring orders of categorical data from category-real ordered pairs
Authors: Chainarong Amornbunchornvej, Navaporn Surasvadi, Anon Plangprasopchok, Suttipong Thajchayapong
Date: 2019-11-15 00:00:00

Given a dataset of careers and incomes, how large a difference of income between any pair of careers would be? Given a dataset of travel time records, how long do we need to spend more when choosing a public transportation mode $A$ instead of $B$ to travel? In this paper, we propose a framework that is able to infer orders of categories as well as magnitudes of difference of real numbers between each pair of categories using Estimation statistics framework. Not only reporting whether an order of categories exists, but our framework also reports the magnitude of difference of each consecutive pairs of categories in the order. In large dataset, our framework is scalable well compared with the existing framework. The proposed framework has been applied to two real-world case studies: 1) ordering careers by incomes based on information of 350,000 households living in Khon Kaen province, Thailand, and 2) ordering sectors by closing prices based on 1060 companies' closing prices of NASDAQ stock markets between years 2000 and 2016. The results of careers ordering show income inequality among different careers. The stock market results illustrate dynamics of sector domination that can change over time. Our approach is able to be applied in any research area that has category-real ordered pairs. Our proposed "Dominant-Distribution Network" provides a novel approach to gain new insight of analyzing category orders. The software of this framework is available for researchers or practitioners within R package: EDOIF.