pwc-archive/papers-with-abstracts huggingface dataset:
Deep Learning of the Eddington Tensor in the Core-collapse Supernova Simulation
Authors: Akira Harada, Shota Nishikawa, Shoichi Yamada
Date: 2021-04-27 00:00:00
Tasks: Relation

We trained deep neural networks (DNNs) as a function of the neutrino energy density, flux, and the fluid velocity to reproduce the Eddington tensor for neutrinos obtained in our first-principles core-collapse supernova (CCSN) simulations. Although the moment method, which is one of the most popular approximations for neutrino transport, requires a closure relation, none of the analytical closure relations commonly employed in the literature captures all aspects of the neutrino angular distribution in momentum space. In this paper, we developed a closure relation by using the DNN that takes the neutrino energy density, flux, and the fluid velocity as the input and the Eddington tensor as the output. We consider two kinds of DNNs: a conventional DNN named a component-wise neural network (CWNN) and a tensor-basis neural network (TBNN). We found that the diagonal component of the Eddington tensor is reproduced better by the DNNs than the M1-closure relation especially for low to intermediate energies. For the off-diagonal component, the DNNs agree better with the Boltzmann solver than the M1 closure at large radii. In the comparison between the two DNNs, the TBNN has slightly better performance than the CWNN. With the new closure relations at hand based on the DNNs that well reproduce the Eddington tensor with much smaller costs, we opened up a new possibility for the moment method.
A Survey on Accuracy-oriented Neural Recommendation: From Collaborative Filtering to Information-rich Recommendation
Authors: Le Wu, Xiangnan He, Xiang Wang, Kun Zhang, Meng Wang
Date: 2021-04-27 00:00:00
Tasks: Collaborative Filtering, Recommendation Systems, Sequential Recommendation

Influenced by the great success of deep learning in computer vision and language understanding, research in recommendation has shifted to inventing new recommender models based on neural networks. In recent years, we have witnessed significant progress in developing neural recommender models, which generalize and surpass traditional recommender models owing to the strong representation power of neural networks. In this survey paper, we conduct a systematic review on neural recommender models from the perspective of recommendation modeling with the accuracy goal, aiming to summarize this field to facilitate researchers and practitioners working on recommender systems. Specifically, based on the data usage during recommendation modeling, we divide the work into collaborative filtering and information-rich recommendation: 1) collaborative filtering, which leverages the key source of user-item interaction data; 2) content enriched recommendation, which additionally utilizes the side information associated with users and items, like user profile and item knowledge graph; and 3) temporal/sequential recommendation, which accounts for the contextual information associated with an interaction, such as time, location, and the past interactions. After reviewing representative work for each type, we finally discuss some promising directions in this field.
The Hessian Screening Rule
Authors: Johan Larsson, Jonas Wallin
Date: 2021-04-27 00:00:00
Tasks: regression

Predictor screening rules, which discard predictors before fitting a model, have had considerable impact on the speed with which sparse regression problems, such as the lasso, can be solved. In this paper we present a new screening rule for solving the lasso path: the Hessian Screening Rule. The rule uses second-order information from the model to provide both effective screening, particularly in the case of high correlation, as well as accurate warm starts. The proposed rule outperforms all alternatives we study on simulated data sets with both low and high correlation for $\ell_1$-regularized least-squares (the lasso) and logistic regression. It also performs best in general on the real data sets that we examine.
Simple yet Sharp Sensitivity Analysis for Unmeasured Confounding
Authors: Jose M. Peña
Date: 2021-04-27 00:00:00
Tasks: Sensitivity

We present a method for assessing the sensitivity of the true causal effect to unmeasured confounding. The method requires the analyst to set two intuitive parameters. Otherwise, the method is assumption-free. The method returns an interval that contains the true causal effect, and whose bounds are arbitrarily sharp, i.e. practically attainable. We show experimentally that our bounds can be tighter than those obtained by the method of Ding and VanderWeele (2016a) which, moreover, requires to set one more parameter than our method. Finally, we extend our method to bound the natural direct and indirect effects when there are measured mediators and unmeasured exposure-outcome confounding.
Node Embedding using Mutual Information and Self-Supervision based Bi-level Aggregation
Authors: Kashob Kumar Roy, Amit Roy, A K M Mahbubur Rahman, M Ashraful Amin, Amin Ahsan Ali
Date: 2021-04-27 00:00:00
Tasks: Node Clustering

Graph Neural Networks (GNNs) learn low dimensional representations of nodes by aggregating information from their neighborhood in graphs. However, traditional GNNs suffer from two fundamental shortcomings due to their local ($l$-hop neighborhood) aggregation scheme. First, not all nodes in the neighborhood carry relevant information for the target node. Since GNNs do not exclude noisy nodes in their neighborhood, irrelevant information gets aggregated, which reduces the quality of the representation. Second, traditional GNNs also fail to capture long-range non-local dependencies between nodes. To address these limitations, we exploit mutual information (MI) to define two types of neighborhood, 1) \textit{Local Neighborhood} where nodes are densely connected within a community and each node would share higher MI with its neighbors, and 2) \textit{Non-Local Neighborhood} where MI-based node clustering is introduced to assemble informative but graphically distant nodes in the same cluster. To generate node presentations, we combine the embeddings generated by bi-level aggregation - local aggregation to aggregate features from local neighborhoods to avoid noisy information and non-local aggregation to aggregate features from non-local neighborhoods. Furthermore, we leverage self-supervision learning to estimate MI with few labeled data. Finally, we show that our model significantly outperforms the state-of-the-art methods in a wide range of assortative and disassortative graphs.
Structure-Aware Hierarchical Graph Pooling using Information Bottleneck
Authors: Kashob Kumar Roy, Amit Roy, A K M Mahbubur Rahman, M Ashraful Amin, Amin Ahsan Ali
Date: 2021-04-27 00:00:00
Tasks: General Classification, Graph Classification

Graph pooling is an essential ingredient of Graph Neural Networks (GNNs) in graph classification and regression tasks. For these tasks, different pooling strategies have been proposed to generate a graph-level representation by downsampling and summarizing nodes' features in a graph. However, most existing pooling methods are unable to capture distinguishable structural information effectively. Besides, they are prone to adversarial attacks. In this work, we propose a novel pooling method named as {HIBPool} where we leverage the Information Bottleneck (IB) principle that optimally balances the expressiveness and robustness of a model to learn representations of input data. Furthermore, we introduce a novel structure-aware Discriminative Pooling Readout ({DiP-Readout}) function to capture the informative local subgraph structures in the graph. Finally, our experimental results show that our model significantly outperforms other state-of-art methods on several graph classification benchmarks and more resilient to feature-perturbation attack than existing pooling methods.
A Review of the Non-Invasive Techniques for Monitoring Different Aspects of Sleep
Authors: Zawar Hussain, Quan Z. Sheng, Wei Emma Zhang, Jorge Ortiz, Seyedamin Pouriyeh
Date: 2021-04-27 00:00:00

Quality sleep is very important for a healthy life. Nowadays, many people around the world are not getting enough sleep which is having negative impacts on their lifestyles. Studies are being conducted for sleep monitoring and have now become an important tool for understanding sleep behavior. The gold standard method for sleep analysis is polysomnography (PSG) conducted in a clinical environment but this method is both expensive and complex for long-term use. With the advancements in the field of sensors and the introduction of off-the-shelf technologies, unobtrusive solutions are becoming common as alternatives for in-home sleep monitoring. Various solutions have been proposed using both wearable and non-wearable methods which are cheap and easy to use for in-home sleep monitoring. In this paper, we present a comprehensive survey of the latest research works (2015 and after) conducted in various categories of sleep monitoring including sleep stage classification, sleep posture recognition, sleep disorders detection, and vital signs monitoring. We review the latest works done using the non-invasive approach and cover both wearable and non-wearable methods. We discuss the design approaches and key attributes of the work presented and provide an extensive analysis based on 10 key factors, to give a comprehensive overview of the recent developments and trends in all four categories of sleep monitoring. We also present some publicly available datasets for different categories of sleep monitoring. In the end, we discuss several open issues and provide future research directions in the area of sleep monitoring.
Discriminative Bayesian filtering lends momentum to the stochastic Newton method for minimizing log-convex functions
Authors: Michael C. Burkhart
Date: 2021-04-27 00:00:00
Tasks: Sequential Bayesian Inference, Stochastic Optimization

To minimize the average of a set of log-convex functions, the stochastic Newton method iteratively updates its estimate using subsampled versions of the full objective's gradient and Hessian. We contextualize this optimization problem as sequential Bayesian inference on a latent state-space model with a discriminatively-specified observation process. Applying Bayesian filtering then yields a novel optimization algorithm that considers the entire history of gradients and Hessians when forming an update. We establish matrix-based conditions under which the effect of older observations diminishes over time, in a manner analogous to Polyak's heavy ball momentum. We illustrate various aspects of our approach with an example and review other relevant innovations for the stochastic Newton method.
Algorithm as Experiment: Machine Learning, Market Design, and Policy Eligibility Rules
Authors: Yusuke Narita, Kohei Yata
Date: 2021-04-26 00:00:00
Tasks: BIG-bench Machine Learning, Decision Making, Selection bias

Algorithms make a growing portion of policy and business decisions. We develop a treatment-effect estimator using algorithmic decisions as instruments for a class of stochastic and deterministic algorithms. Our estimator is consistent and asymptotically normal for well-defined causal effects. A special case of our setup is multidimensional regression discontinuity designs with complex boundaries. We apply our estimator to evaluate the Coronavirus Aid, Relief, and Economic Security Act, which allocated many billions of dollars worth of relief funding to hospitals via an algorithmic rule. The funding is shown to have little effect on COVID-19-related hospital activities. Naive estimates exhibit selection bias.
Continual Distributed Learning for Crisis Management
Authors: Aman Priyanshu, Mudit Sinha, Shreyans Mehta
Date: 2021-04-26 00:00:00
Tasks: Continual Learning, Disaster Response, Federated Learning, Management, text-classification, Text Classification

Social media platforms such as Twitter, Facebook etc can be utilised as an important source of information during disaster events. This information can be used for disaster response and crisis management if processed accurately and quickly. However, the data present in such situations is ever-changing, and using considerable resources during such a crisis is not feasible. Therefore, we have to develop a low resource and continually learning system that incorporates text classification models which are robust against noisy and unordered data. We utilised Distributed learning which enabled us to learn on resource-constrained devices, then to alleviate catastrophic forgetting in our target neural networks we utilized regularization. We then applied federated averaging for distributed learning and to aggregate the central model for continual learning.
Multi-Task Learning for End-to-End ASR Word and Utterance Confidence with Deletion Prediction
Authors: David Qiu, Yanzhang He, Qiujia Li, Yu Zhang, Liangliang Cao, Ian McGraw
Date: 2021-04-26 00:00:00
Tasks: Automatic Speech Recognition, Automatic Speech Recognition (ASR), Multi-Task Learning, speech-recognition, Speech Recognition

Confidence scores are very useful for downstream applications of automatic speech recognition (ASR) systems. Recent works have proposed using neural networks to learn word or utterance confidence scores for end-to-end ASR. In those studies, word confidence by itself does not model deletions, and utterance confidence does not take advantage of word-level training signals. This paper proposes to jointly learn word confidence, word deletion, and utterance confidence. Empirical results show that multi-task learning with all three objectives improves confidence metrics (NCE, AUC, RMSE) without the need for increasing the model size of the confidence estimation module. Using the utterance-level confidence for rescoring also decreases the word error rates on Google's Voice Search and Long-tail Maps datasets by 3-5% relative, without needing a dedicated neural rescorer.
Semantic Analysis for Automated Evaluation of the Potential Impact of Research Articles
Authors: Neslihan Suzen, Alexander Gorban, Jeremy Levesley, Evgeny Mirkes
Date: 2021-04-26 00:00:00
Tasks: Articles, Citation Prediction, General Classification, text-classification, Text Classification

Can the analysis of the semantics of words used in the text of a scientific paper predict its future impact measured by citations? This study details examples of automated text classification that achieved 80% success rate in distinguishing between highly-cited and little-cited articles. Automated intelligent systems allow the identification of promising works that could become influential in the scientific community. The problems of quantifying the meaning of texts and representation of human language have been clear since the inception of Natural Language Processing. This paper presents a novel method for vector representation of text meaning based on information theory and show how this informational semantics is used for text classification on the basis of the Leicester Scientific Corpus. We describe the experimental framework used to evaluate the impact of scientific articles through their informational semantics. Our interest is in citation classification to discover how important semantics of texts are in predicting the citation count. We propose the semantics of texts as an important factor for citation prediction. For each article, our system extracts the abstract of paper, represents the words of the abstract as vectors in Meaning Space, automatically analyses the distribution of scientific categories (Web of Science categories) within the text of abstract, and then classifies papers according to citation counts (highly-cited, little-cited). We show that an informational approach to representing the meaning of a text has offered a way to effectively predict the scientific impact of research papers.
Multi-Output Random Forest Regression to Emulate the Earliest Stages of Planet Formation
Authors: Kevin Hoffman, Jae Yoon Sung, André Zazzera
Date: 2021-04-26 00:00:00
Tasks: regression

In the current paradigm of planet formation research, it is believed that the first step to forming massive bodies (such as asteroids and planets) requires that small interstellar dust grains floating through space collide with each other and grow to larger sizes. The initial formation of these pebbles is governed by an integro-differential equation known as the Smoluchowski coagulation equation, to which analytical solutions are intractable for all but the simplest possible scenarios. While brute-force methods of approximation have been developed, they are computationally costly, currently making it infeasible to simulate this process including other physical processes relevant to planet formation, and across the very large range of scales on which it occurs. In this paper, we take a machine learning approach to designing a system for a much faster approximation. We develop a multi-output random forest regression model trained on brute-force simulation data to approximate distributions of dust particle sizes in protoplanetary disks at different points in time. The performance of our random forest model is measured against the existing brute-force models, which are the standard for realistic simulations. Results indicate that the random forest model can generate highly accurate predictions relative to the brute-force simulation results, with an $R^{2}$ of 0.97, and do so significantly faster than brute-force methods.
LCS-DIVE: An Automated Rule-based Machine Learning Visualization Pipeline for Characterizing Complex Associations in Classification
Authors: Robert Zhang, Rachael Stolzenberg-Solomon, Shannon M. Lynch, Ryan J. Urbanowicz
Date: 2021-04-26 00:00:00
Tasks: BIG-bench Machine Learning, Clustering, Fairness, Feature Importance

Machine learning (ML) research has yielded powerful tools for training accurate prediction models despite complex multivariate associations (e.g. interactions and heterogeneity). In fields such as medicine, improved interpretability of ML modeling is required for knowledge discovery, accountability, and fairness. Rule-based ML approaches such as Learning Classifier Systems (LCSs) strike a balance between predictive performance and interpretability in complex, noisy domains. This work introduces the LCS Discovery and Visualization Environment (LCS-DIVE), an automated LCS model interpretation pipeline for complex biomedical classification. LCS-DIVE conducts modeling using a new scikit-learn implementation of ExSTraCS, an LCS designed to overcome noise and scalability in biomedical data mining yielding human readable IF:THEN rules as well as feature-tracking scores for each training sample. LCS-DIVE leverages feature-tracking scores and/or rules to automatically guide characterization of (1) feature importance (2) underlying additive, epistatic, and/or heterogeneous patterns of association, and (3) model-driven heterogeneous instance subgroups via clustering, visualization generation, and cluster interrogation. LCS-DIVE was evaluated over a diverse set of simulated genetic and benchmark datasets encoding a variety of complex multivariate associations, demonstrating its ability to differentiate between them and then applied to characterize associations within a real-world study of pancreatic cancer.
AdaGNN: Graph Neural Networks with Adaptive Frequency Response Filter
Authors: Yushun Dong, Kaize Ding, Brian Jalaian, Shuiwang Ji, Jundong Li
Date: 2021-04-26 00:00:00
Tasks: Denoising, Graph Learning, Graph Neural Network, Representation Learning

Graph Neural Networks have recently become a prevailing paradigm for various high-impact graph analytical problems. Existing efforts can be mainly categorized as spectral-based and spatial-based methods. The major challenge for the former is to find an appropriate graph filter to distill discriminative information from input signals for learning. Recently, myriads of explorations are made to achieve better graph filters, e.g., Graph Convolutional Network (GCN), which leverages Chebyshev polynomial truncation to seek an approximation of graph filters and bridge these two families of methods. Nevertheless, it has been shown in recent studies that GCN and its variants are essentially employing fixed low-pass filters to perform information denoising. Thus their learning capability is rather limited and may over-smooth node representations at deeper layers. To tackle these problems, we develop a novel graph neural network framework AdaGNN with a well-designed adaptive frequency response filter. At its core, AdaGNN leverages a simple but elegant trainable filter that spans across multiple layers to capture the varying importance of different frequency components for node representation learning. The inherent differences among different feature channels are also well captured by the filter. As such, it empowers AdaGNN with stronger expressiveness and naturally alleviates the over-smoothing problem. We empirically validate the effectiveness of the proposed framework on various benchmark datasets. Theoretical analysis is also provided to show the superiority of the proposed AdaGNN. The open-source implementation of AdaGNN can be found here: https://github.com/yushundong/AdaGNN.
Learning-based decentralized offloading decision making in an adversarial environment
Authors: Byungjin Cho, Yu Xiao
Date: 2021-04-26 00:00:00
Tasks: Cloud Computing, Decision Making

Vehicular fog computing (VFC) pushes the cloud computing capability to the distributed fog nodes at the edge of the Internet, enabling compute-intensive and latency-sensitive computing services for vehicles through task offloading. However, a heterogeneous mobility environment introduces uncertainties in terms of resource supply and demand, which are inevitable bottlenecks for the optimal offloading decision. Also, these uncertainties bring extra challenges to task offloading under the oblivious adversary attack and data privacy risks. In this article, we develop a new adversarial online learning algorithm with bandit feedback based on the adversarial multi-armed bandit theory, to enable scalable and low-complexity offloading decision making. Specifically, we focus on optimizing fog node selection with the aim of minimizing the offloading service costs in terms of delay and energy. The key is to implicitly tune the exploration bonus in the selection process and the assessment rules of the designed algorithm, taking into account volatile resource supply and demand. We theoretically prove that the input-size dependent selection rule allows to choose a suitable fog node without exploring the sub-optimal actions, and also an appropriate score patching rule allows to quickly adapt to evolving circumstances, which reduce variance and bias simultaneously, thereby achieving a better exploitation-exploration balance. Simulation results verify the effectiveness and robustness of the proposed algorithm.