pwc-archive/papers-with-abstracts huggingface dataset:
Improving Prediction of Cognitive Performance using Deep Neural Networks in Sparse Data
Authors: Sharath Koorathota, Arunesh Mittal, Richard P. Sloan, Paul Sajda
Date: 2021-12-28 00:00:00

Cognition in midlife is an important predictor of age-related mental decline and statistical models that predict cognitive performance can be useful for predicting decline. However, existing models struggle to capture complex relationships between physical, sociodemographic, psychological and mental health factors that effect cognition. Using data from an observational, cohort study, Midlife in the United States (MIDUS), we modeled a large number of variables to predict executive function and episodic memory measures. We used cross-sectional and longitudinal outcomes with varying sparsity, or amount of missing data. Deep neural network (DNN) models consistently ranked highest in all of the cognitive performance prediction tasks, as assessed with root mean squared error (RMSE) on out-of-sample data. RMSE differences between DNN and other model types were statistically significant (T(8) = -3.70; p < 0.05). The interaction effect between model type and sparsity was significant (F(9)=59.20; p < 0.01), indicating the success of DNNs can partly be attributed to their robustness and ability to model hierarchical relationships between health-related factors. Our findings underscore the potential of neural networks to model clinical datasets and allow better understanding of factors that lead to cognitive decline.
Ensemble Recognition in Reproducing Kernel Hilbert Spaces through Aggregated Measurements
Authors: Wei Miao, Gong Cheng, Jr-Shin Li
Date: 2021-12-28 00:00:00

In this paper, we study the problem of learning dynamical properties of ensemble systems from their collective behaviors using statistical approaches in reproducing kernel Hilbert space (RKHS). Specifically, we provide a framework to identify and cluster multiple ensemble systems through computing the maximum mean discrepancy (MMD) between their aggregated measurements in an RKHS, without any prior knowledge of the system dynamics of ensembles. Then, leveraging the gradient flow of the newly proposed notion of aggregated Markov parameters, we present a systematic framework to recognize and identify an ensemble systems using their linear approximations. Finally, we demonstrate that the proposed approaches can be extended to cluster multiple unknown ensembles in RKHS using their aggregated measurements. Numerical experiments show that our approach is reliable and robust to ensembles with different types of system dynamics.
A Transformer-based Approach for Translating Natural Language to Bash Commands
Authors: Quchen Fu, Zhongwei Teng, Jules White, Douglas C. Schmidt
Date: 2021-12-14 00:00:00
Proceeding: 20th IEEE International Conference on Machine Learning and Applications - ICMLA 2021 12
Tasks: Code Translation, Sentence, Translation

This paper explores the translation of natural language into Bash Commands, which developers commonly use to accomplish command-line tasks in a terminal. In our approach a terminal takes a command as a sentence in plain English and translates it into the corresponding string of Bash Commands. The paper analyzes the performance of several architectures on this translation problem using the data from the NLC2CMD competition at the NeurIPS 2020 conference. The approach presented in this paper is the best performing architecture on this problem to date and improves the current state-of-the-art accuracy on this translation task from 13.8% to 53.2%.
Dynamic Time Warping Clustering to Discover Socio-Economic Characteristics in Smart Water Meter Data
Authors: D. B. Steffelbauer, E. J. M. Blokker, S. G. Buchberger, A. Knobbe, E. Abraham
Date: 2021-12-27 00:00:00
Tasks: Clustering, Dynamic Time Warping

Socio-economic characteristics are influencing the temporal and spatial variability of water demand - the biggest source of uncertainties within water distribution system modeling. Improving our knowledge on these influences can be utilized to decrease demand uncertainties. This paper aims to link smart water meter data to socio-economic user characteristics by applying a novel clustering algorithm that uses dynamic time warping on daily demand patterns. The approach is tested on simulated and measured single family home datasets. We show that the novel algorithm performs better compared to commonly used clustering methods, both, in finding the right number of clusters as well as assigning patterns correctly. Additionally, the methodology can be used to identify outliers within clusters of demand patterns. Furthermore, this study investigates which socio-economic characteristics (e.g. employment status, number of residents) are prevalent within single clusters and, consequently, can be linked to the shape of the cluster's barycenters. In future, the proposed methods in combination with stochastic demand models can be used to fill data-gaps in hydraulic models.
REAL-TIME DROWSINESS DETECTION (RTD2)
Authors: Sri Krishna H V, Tarun Srinivas, Kedarnath R Gubbi, Vijay Kumar S
Date: 2021-09-04 00:00:00
Proceeding: HUST Journal 2021 9

In the present era, where the roads are filled with vehicles all the time, accidents are 
prone to happen. Studies show that over 50% of these road accidents in a year, are due to driver 
drowsiness and fatigue. Many attempts have been made to develop a solution to this problem to 
detect when the driver is drowsy, but everything has its own set of limitations. Our project aims to 
solve this problem by detecting the drowsiness of the driver and sending an alert. We propose to use 
the Dlib Library to detect the driver’s face, and a number of landmarks are plotted on the entire 
face of the driver. Using these landmarks, various calculations and comparisons take place to check 
whether the drivers’ eyes are closed or open, and whether the driver is yawning or not. From this, 
the system determines whether the driver is drowsy or not. If the driver is found to be drowsy, an 
audio and visual alert is triggered to alert the driver. With this project, we aim to help society 
by reducing road accidents, and saving lives and property.
Nested Nonparametric Instrumental Variable Regression
Authors: Isaac Meza, Rahul Singh
Date: 2021-12-28 00:00:00
Tasks: BIG-bench Machine Learning, Causal Inference, Econometrics, regression

Several causal parameters in short panel data models are functionals of a nested nonparametric instrumental variable regression (nested NPIV). Recent examples include mediated, time varying, and long term treatment effects identified using proxy variables. In econometrics, examples arise in triangular simultaneous equations and hedonic price systems. However, it appears that explicit mean square convergence rates for nested NPIV are unknown, preventing inference on some of these parameters with generic machine learning. A major challenge is compounding ill posedness due to the nested inverse problems. To limit how ill posedness compounds, we introduce two techniques: relative well posedness, and multiple robustness to ill posedness. With these techniques, we provide explicit mean square rates for nested NPIV and efficient inference for recently identified causal parameters. Our nonasymptotic analysis accommodates neural networks, random forests, and reproducing kernel Hilbert spaces. It extends to causal functions, e.g. heterogeneous long term treatment effects.
GREED: A Neural Framework for Learning Graph Distance Functions
Authors: Rishabh Ranjan, Siddharth Grover, Sourav Medya, Venkatesan Chakaravarthy, Yogish Sabharwal, Sayan Ranu
Date: 2021-12-24 00:00:00
Tasks: CPU, GPU, Graph Neural Network, Graph Similarity, Inductive Bias, Retrieval

Among various distance functions for graphs, graph and subgraph edit distances (GED and SED respectively) are two of the most popular and expressive measures. Unfortunately, exact computations for both are NP-hard. To overcome this computational bottleneck, neural approaches to learn and predict edit distance in polynomial time have received much interest. While considerable progress has been made, there exist limitations that need to be addressed. First, the efficacy of an approximate distance function lies not only in its approximation accuracy, but also in the preservation of its properties. To elaborate, although GED is a metric, its neural approximations do not provide such a guarantee. This prohibits their usage in higher order tasks that rely on metric distance functions, such as clustering or indexing. Second, several existing frameworks for GED do not extend to SED due to SED being asymmetric. In this work, we design a novel siamese graph neural network called GREED, which through a carefully crafted inductive bias, learns GED and SED in a property-preserving manner. Through extensive experiments across 10 real graph datasets containing up to 7 million edges, we establish that GREED is not only more accurate than the state of the art, but also up to 3 orders of magnitude faster. Even more significantly, due to preserving the triangle inequality, the generated embeddings are indexable and consequently, even in a CPU-only environment, GREED is up to 50 times faster than GPU-powered baselines for graph / subgraph retrieval.
Non-Convex Joint Community Detection and Group Synchronization via Generalized Power Method
Authors: Sijin Chen, Xiwei Cheng, Anthony Man-Cho So
Date: 2021-12-28 00:00:00
Tasks: Community Detection, Stochastic Block Model

This paper proposes a Generalized Power Method (GPM) to tackle the problem of community detection and group synchronization simultaneously in a direct non-convex manner. Under the stochastic group block model (SGBM), theoretical analysis indicates that the algorithm is able to exactly recover the ground truth in $O(n\log^2n)$ time, sharply outperforming the benchmark method of semidefinite programming (SDP) in $O(n^{3.5})$ time. Moreover, a lower bound of parameters is given as a necessary condition for exact recovery of GPM. The new bound breaches the information-theoretic threshold for pure community detection under the stochastic block model (SBM), thus demonstrating the superiority of our simultaneous optimization algorithm over the trivial two-stage method which performs the two tasks in succession. We also conduct numerical experiments on GPM and SDP to evidence and complement our theoretical analysis.
Transfer learning of phase transitions in percolation and directed percolation
Authors: Jianmin Shen, Feiyi Liu, Shiyang Chen, Dian Xu, Xiangna Chen, Shengfeng Deng, Wei Li, Gabor Papp, Chunbin Yang
Date: 2021-12-31 00:00:00
Tasks: Transfer Learning

The latest advances of statistical physics have shown remarkable performance of machine learning in identifying phase transitions. In this paper, we apply domain adversarial neural network (DANN) based on transfer learning to studying non-equilibrium and equilibrium phase transition models, which are percolation model and directed percolation (DP) model, respectively. With the DANN, only a small fraction of input configurations (2d images) needs to be labeled, which is automatically chosen, in order to capture the critical point. To learn the DP model, the method is refined by an iterative procedure in determining the critical point, which is a prerequisite for the data collapse in calculating the critical exponent $\nu_{\perp}$. We then apply the DANN to a two-dimensional site percolation with configurations filtered to include only the largest cluster which may contain the information related to the order parameter. The DANN learning of both models yields reliable results which are comparable to the ones from Monte Carlo simulations. Our study also shows that the DANN can achieve quite high accuracy at much lower cost, compared to the supervised learning.
A Neural Network Solves, Explains, and Generates University Math Problems by Program Synthesis and Few-Shot Learning at Human Level
Authors: Iddo Drori, Sarah Zhang, Reece Shuttleworth, Leonard Tang, Albert Lu, Elizabeth Ke, Kevin Liu, Linda Chen, Sunny Tran, Newman Cheng, Roman Wang, Nikhil Singh, Taylor L. Patti, Jayson Lynch, Avi Shporer, Nakul Verma, Eugene Wu, Gilbert Strang
Date: 2021-12-31 00:00:00
Tasks: Few-Shot Learning, Language Modelling, Math, Mathematical Reasoning, Program Synthesis, Zero-Shot Learning

We demonstrate that a neural network pre-trained on text and fine-tuned on code solves mathematics course problems, explains solutions, and generates new questions at a human level. We automatically synthesize programs using few-shot learning and OpenAI's Codex transformer and execute them to solve course problems at 81% automatic accuracy. We curate a new dataset of questions from MIT's largest mathematics courses (Single Variable and Multivariable Calculus, Differential Equations, Introduction to Probability and Statistics, Linear Algebra, and Mathematics for Computer Science) and Columbia University's Computational Linear Algebra. We solve questions from a MATH dataset (on Prealgebra, Algebra, Counting and Probability, Intermediate Algebra, Number Theory, and Precalculus), the latest benchmark of advanced mathematics problems designed to assess mathematical reasoning. We randomly sample questions and generate solutions with multiple modalities, including numbers, equations, and plots. The latest GPT-3 language model pre-trained on text automatically solves only 18.8% of these university questions using zero-shot learning and 30.8% using few-shot learning and the most recent chain of thought prompting. In contrast, program synthesis with few-shot learning using Codex fine-tuned on code generates programs that automatically solve 81% of these questions. Our approach improves the previous state-of-the-art automatic solution accuracy on the benchmark topics from 8.8% to 81.1%. We perform a survey to evaluate the quality and difficulty of generated questions. This work is the first to automatically solve university-level mathematics course questions at a human level and the first work to explain and generate university-level mathematics course questions at scale, a milestone for higher education.
Deep Graph Clustering via Dual Correlation Reduction
Authors: Yue Liu, Wenxuan Tu, Sihang Zhou, Xinwang Liu, Linxuan Song, Xihong Yang, En Zhu
Date: 2021-12-29 00:00:00
Tasks: Clustering, Feature Correlation, Graph Clustering

Deep graph clustering, which aims to reveal the underlying graph structure and divide the nodes into different groups, has attracted intensive attention in recent years. However, we observe that, in the process of node encoding, existing methods suffer from representation collapse which tends to map all data into the same representation. Consequently, the discriminative capability of the node representation is limited, leading to unsatisfied clustering performance. To address this issue, we propose a novel self-supervised deep graph clustering method termed Dual Correlation Reduction Network (DCRN) by reducing information correlation in a dual manner. Specifically, in our method, we first design a siamese network to encode samples. Then by forcing the cross-view sample correlation matrix and cross-view feature correlation matrix to approximate two identity matrices, respectively, we reduce the information correlation in the dual-level, thus improving the discriminative capability of the resulting features. Moreover, in order to alleviate representation collapse caused by over-smoothing in GCN, we introduce a propagation regularization term to enable the network to gain long-distance information with the shallow network structure. Extensive experimental results on six benchmark datasets demonstrate the effectiveness of the proposed DCRN against the existing state-of-the-art methods.
Finding the Task-Optimal Low-Bit Sub-Distribution in Deep Neural Networks
Authors: Runpei Dong, Zhanhong Tan, Mengdi Wu, Linfeng Zhang, Kaisheng Ma
Date: 2021-12-30 00:00:00
Tasks: CPU, image-classification, Image Classification, Model Compression, object-detection, Object Detection, Quantization

Quantized neural networks typically require smaller memory footprints and lower computation complexity, which is crucial for efficient deployment. However, quantization inevitably leads to a distribution divergence from the original network, which generally degrades the performance. To tackle this issue, massive efforts have been made, but most existing approaches lack statistical considerations and depend on several manual configurations. In this paper, we present an adaptive-mapping quantization method to learn an optimal latent sub-distribution that is inherent within models and smoothly approximated with a concrete Gaussian Mixture (GM). In particular, the network weights are projected in compliance with the GM-approximated sub-distribution. This sub-distribution evolves along with the weight update in a co-tuning schema guided by the direct task-objective optimization. Sufficient experiments on image classification and object detection over various modern architectures demonstrate the effectiveness, generalization property, and transferability of the proposed method. Besides, an efficient deployment flow for the mobile CPU is developed, achieving up to 7.46$\times$ inference acceleration on an octa-core ARM CPU. Our codes have been publicly released at \url{https://github.com/RunpeiDong/DGMS}.
Importance of Empirical Sample Complexity Analysis for Offline Reinforcement Learning
Authors: Samin Yeasar Arnob, Riashat Islam, Doina Precup
Date: 2021-12-31 00:00:00
Tasks: Offline RL, reinforcement-learning, Reinforcement Learning, Reinforcement Learning (RL)

We hypothesize that empirically studying the sample complexity of offline reinforcement learning (RL) is crucial for the practical applications of RL in the real world. Several recent works have demonstrated the ability to learn policies directly from offline data. In this work, we ask the question of the dependency on the number of samples for learning from offline data. Our objective is to emphasize that studying sample complexity for offline RL is important, and is an indicator of the usefulness of existing offline algorithms. We propose an evaluation approach for sample complexity analysis of offline RL.
Lunatic Stocks: Moon Phases as Irregular Sampling Features for Pattern Recognition in the Stock Markets
Authors: Luis A. Mateos
Date: 2021-12-14 00:00:00

This paper presents a novel idea on incorporating the Moon phases to the classic Gregorian (Solar) calendar time sampling methods for finding meaningful patterns in the stock markets. The four main Moon phases (New Moon, First quarter, Full Moon and Third quarter) are irregular in time but with well defined sampling structure as the Moon orbits the Earth completing its period. A Full Moon may appear in one month of the year on the 2nd, on the next month the Full moon may appear on the 4th and in the next ten years on the 13th of the same month. This structure which is irregular in time makes it interesting to study together with the stock market data. Moreover, the moon affects multiple physical things on the earth, such as the ocean tides, the behavior of living organisms as well as humans mood and decision when risking and investing.
Attention-based Bidirectional LSTM for Deceptive Opinion Spam Classification
Authors: Ashish Salunkhe
Date: 2021-12-29 00:00:00
Tasks: Classification, Decision Making

Online Reviews play a vital role in e commerce for decision making. Much of the population makes the decision of which places, restaurant to visit, what to buy and from where to buy based on the reviews posted on the respective platforms. A fraudulent review or opinion spam is categorized as an untruthful or deceptive review. Positive reviews of a product or a restaurant helps attract customers and thereby lead to an increase in sales whereas negative reviews may hamper the progress of a restaurant or sales of a product and thereby lead to defamed reputation and loss. Fraudulent reviews are deliberately posted on various online review platforms to trick customers to buy, visit or distract against a product or a restaurant. They are also written to commend or discredit the product's repute. The work aims at detecting and classifying the reviews as deceptive or truthful. It involves use of various deep learning techniques for classifying the reviews and an overview of proposed approach involving Attention based Bidirectional LSTM to tackle issues related to semantic information in reviews and a comparative study over baseline machine learning techniques for review classification.
Fast Learning of MNL Model from General Partial Rankings with Application to Network Formation Modeling
Authors: Jiaqi Ma, Xingjian Zhang, Qiaozhu Mei
Date: 2021-12-31 00:00:00
Tasks: Discrete Choice Models, parameter estimation

Multinomial Logit (MNL) is one of the most popular discrete choice models and has been widely used to model ranking data. However, there is a long-standing technical challenge of learning MNL from many real-world ranking data: exact calculation of the MNL likelihood of \emph{partial rankings} is generally intractable. In this work, we develop a scalable method for approximating the MNL likelihood of general partial rankings in polynomial time complexity. We also extend the proposed method to learn mixture of MNL. We demonstrate that the proposed methods are particularly helpful for applications to choice-based network formation modeling, where the formation of new edges in a network is viewed as individuals making choices of their friends over a candidate set. The problem of learning mixture of MNL models from partial rankings naturally arises in such applications. And the proposed methods can be used to learn MNL models from network data without the strong assumption that temporal orders of all the edge formation are available. We conduct experiments on both synthetic and real-world network data to demonstrate that the proposed methods achieve more accurate parameter estimation and better fitness of data compared to conventional methods.
Frame invariance and scalability of neural operators for partial differential equations
Authors: Muhammad I. Zafar, Jiequn Han, Xu-Hui Zhou, Heng Xiao
Date: 2021-12-28 00:00:00

Partial differential equations (PDEs) play a dominant role in the mathematical modeling of many complex dynamical processes. Solving these PDEs often requires prohibitively high computational costs, especially when multiple evaluations must be made for different parameters or conditions. After training, neural operators can provide PDEs solutions significantly faster than traditional PDE solvers. In this work, invariance properties and computational complexity of two neural operators are examined for transport PDE of a scalar quantity. Neural operator based on graph kernel network (GKN) operates on graph-structured data to incorporate nonlocal dependencies. Here we propose a modified formulation of GKN to achieve frame invariance. Vector cloud neural network (VCNN) is an alternate neural operator with embedded frame invariance which operates on point cloud data. GKN-based neural operator demonstrates slightly better predictive performance compared to VCNN. However, GKN requires an excessively high computational cost that increases quadratically with the increasing number of discretized objects as compared to a linear increase for VCNN.